Chapter 5 – Conclusions
Both the small number of students and potential interference by the interventionist means that the results cannot be transferred to larger groups of students and conclusions cannot be drawn that are applicable to other settings. Therefore, no direct conclusion can be drawn based on this study in answer the research questions ‘Does the use of high quality instructional videos have an effect on the intervention learning of mathematics students?’ and ‘Is it more effective to use direct instruction or instructional videos to address student needs?’. However, data did indicate that the use of KAC, which included instructional videos in its program, did lead to higher retention by participating mathematics students. The concept of Five in a Row also appeared to add rigor and accountability.
Upon reflection, there were marked differences between the control and experimental treatments that may have effected why the experimental group retained more knowledge than the control. There was of course the instruction itself. The interventionist may have provided different content instruction or inferior instruction to that found on the video. However, instruction in terms of content largely mirrored what was taught on the videos. There were additional differences between control and experimental treatment: different worksheets, different type and number of practice problems, the video could be individually controlled, and additional discussion occurred during the experimental treatment.
The experimental treatment had worksheets to be completed during the video. The videos provided strategies one after the other. Students took notes on the video and did not actually complete any problems on their own until after they had watched all the strategies (See F2: Experimental Worksheet in Appendix F). The control worksheets were set up differently. The interventionist taught a strategy, the student practiced the strategy; the interventionist taught another strategy, the student practiced the strategy, etc. until the student moved on to the independent practice questions (See F1: Control Worksheet in Appendix F).
The KAC Five in a Row problems’ were more numerous and held the students more accountable. Students had to answer correctly, with no hints, a minimum of 5 problems to finish the lesson, thus ensuring students could complete the problems independently and consistently. In the control treatment, the students only had to complete 4-5 practice problems for each topic. If students got the problem wrong, they would receive feedback from the interventionist and then fix it; they did not have to complete more problems. This perhaps led to lower expectations as they could always go back and fix the problem if they got it wrong. Making an error had no real effect on them, unlike with Khan Academy. They also because of this completed a lower number of problems and had less of a chance to practice over a longer period of time.
Experimental practice was more rigorous, as it used larger numbers, while the control problems tended to use smaller numbers. On the assessment students who received the control treatment in mixed numbers and improper fractions scored on average, 50 percent and 16.5 percent respectively. In comparison, students who received the more rigorous experimental treatment scored an average of 75 percent on both topics, demonstrating higher long term retention. In the control practice problems, the logic was using non-intimidating numbers would be less overwhelming and would lead students to focus more on the process.
It could be argued that lower level math classes do the same, providing easier numbers for students to work with because the focus is on the process. This opens the question: would these classes, and students, benefit from struggling with larger numbers while solving grade level work than doing the same grade level work with smaller numbers? Does it put more of a stress on the brain with the computations required leading to higher memory recall? Does the challenge of the computations (which arguably used more focus than easier problems), perhaps due to the fact that students had to work harder at it, lead to better memory and recall? This question and line of thinking corresponds with the idea that struggle is good for students, as struggle leads to frustration and “such frustration is a precursor to deep, lasting learning.” (Ginsburg, 2012). When students struggle, they also come up with their own strategies for solving problems. When this strategy making is observed and the interventionist is available to step in and redirect, the students benefit by gaining skills they can use in the future.
An important advantage of the video was that it could be re-watched and paused. During the video the students completed a worksheet, taking notes during specifically identified parts of the video. The students felt it was hard to take notes and watch at the same time, as expressed in the interview. However because it was a video, the students were able to “stop it and then write” or “rewind it and then go over everything”. The video provided distance which led to more openness during the study about how much they liked it.
There were also variations in discussion questions. There were consistently discussion questions after students watched the videos. When there were questions in the control, they differed from the questions asked in the experimental treatment. An example of this is in the improper fractions lesson. The control lesson had no discussion questions while the experimental had asked “What are the two ways he solves the problem? How would you solve the problem?” and “What is similar and what is different between changing from a mixed number to an improper fraction and from an improper fractions to a mixed number?”
The results of this study would suggest a more formal study with a larger population and outside evaluation would be appropriate. Repetition of the study, even with in the same circumstances, could provide additional testing for the conclusion. Adjustment to some of the variables in direct instruction should be made in order to more closely align to the experimental treatment. More effort should be made for an even split between the control group and the experimental treatments. The study could be expanded to include more Tier III and possibly Tier II students who work with other interventionist or expand the study to a full semester or academic year time frame.
Outcomes from a more rigorous study which could add to the research literature in this area include the effect instructional videos have on learning, the effect Khan Academy Coach has on learning, and the effect rigorous, challenging, and accountable practice has to potentially lead to higher levels of retention.
One question that arose for further study was whether a higher level of practice leads to a higher level of retention. More study on these topics is needed. Although students were proficient in a skill on a post-test, on the post-assessment, many students struggled to remember all the steps to solving a problem and struggled to differentiate between skills (i.e. multiples vs. factors; least common multiple v. greatest common factor). Questions that arose in terms of retention included: What methods while instructing promote long term retention? What effective strategies can be used over time to reinforce and revisit previous learned skills? Another issue that arose was differentiating between skills, as mentioned. Students particularly struggled with the difference between multiples of a number and factors of a number (even if, interestingly, they were able to correctly find the LCM and GCF of a pair of numbers). Students may tend to learn the process that goes with certain terms but when faced with independent terms (such as multiple) they are stymied. For future research, it would be interesting to use several different methods for teaching these terms, and see what effect each method has on long term retention and understanding.
Through this study, students were provided with instruction fidelity and a strong basis for 6th and 7th grade math, ongoing into the rest of the school year. Tools used in this study that will continued to be used to the benefit of the instructor and students are: instructional videos, Five in a Row tool, teacher log, pre and post-tests, and pre and post-assessments. The instructional videos provided instruction that was of quality and led to long term retention. The Five in a Row tool provided students with accountable, rigorous practice. The teacher log helped the interventionist keep detailed record of student progress, issues that arose, and the success/personal feedback from students about lessons. The pre and post-tests helped track student’s knowledge before and after each lesson, ensuring appropriate growth. The pre-and post-assessments measured students’ retention of topics over a longer period of time and highlighted any areas that needed to be reviewed or retaught, as well as lessons that needed to be reformulated to ensure for better long term retention.
The next steps for this research and the researcher are to look more closely at whether instructional videos or how practice effects retention. Additionally, instructional videos from different sites can be tested to see if students learn better with instructional videos as an overall trend. Another topic of further research to look at the effect struggle has on retention. If I were to repeat this study, I would change the quantity and difficulty level of practice problems for the control group, ensure an even split between the control group and the experimental groups lessons, and expand the study to include more Tier III and possible Tier II students working with other interventionists.
Through this study, I have come to the conclusion SRBI is not just for helping students with their math when they struggle. It is to teach these students to struggle and to build their understanding that struggling with math is a natural and desired reaction and not something scary and intimidating. Additionally, it is time where they can learn to become more independent in their mathematics and build their mathematical strategies. The interventionist is able to observe the students learning abilities, learning type, and patterns and form instruction from there on a very personal level. The purpose of the interventionist is to “carefully choos[e] places to support them while keeping them in control of processing”, not to help students in everything but rather than to serve as a guide leading to independence in their math abilities (Johnston, 2010, p. 604).