(0) Introduction and links to the other teacher training topics
Assessment before teaching with timely practice, enables the teacher to teach on firm learning foundations (see planTeaching) and the app to keep what the learner can already remember fresh (see retrievalPracticeTheory). Feedback of the type - tick or not - will strengthen the chunks in the learners' long term memory (see chunk-basedTheory).
Assessment during the teach part of the lesson enables the teacher to teach more effectively i.e. give instant feedback. Self assessment by the learner during the practice part of the lesson enables the learner to know to seek help from a teacher or their peers i.e. feedback-dialogue.
Assessment the next lesson, after teaching, enables the teacher, app and learner to work together to keep the learning fresh and gradually stretch the durability of recall-ability of new and existing learning. The time set aside for pre assess and retrieval practice additionally gives the teacher and learner the time and ability to do feedback-dialogue when errors are made. After a couple of months, the retrieval practice, scheduled by the app, means the teacher will almost always find the teaching of each lesson has become deeply embedded in long term memory and the teacher can teach more on the topic on the firm learning foundations.
(1) Always teach on firm learning foundations = mastery learning
Effective pre assess, finding out what learners "already know" means we can make best use of lesson time. We won't teach learners what they "already know", nor attempt to teach learners skills for which they have insufficiently firm learning foundations, instead we can teach in the "sweet spot" between. So teaching and learning become more efficient i.e. we can increase the output (retained learning) per hour (of lesson time).
With timely practice, teachers only need to pre assess a topic once, before they teach the topic for the first time. Next time the teacher spirals around to teach the topic, the teacher will be able to see in fine detail what the learners already know/don't know yet from the assessment of retrieval practice data the app collects.
Of course it is no good collecting robust assessment for learning data, unless the teacher uses it to plan teaching.
| Expand |
|---|
| title | Why find firm learning foundations before teaching with timely practice |
|---|
|
When the teacher finds each learners firm learning foundations on a topic and uses that data to decide where to start teaching that topic, the teacher can be confident that the learners will retain new teaching because the teacher is teaching on firm learning foundations, the app schedules a retrieval practice programme for each bite (layer) of new learning and that when feedback needs to be given, it has the best chance of success, because the teacher and learner are entering into a feedback-dialogue and the app adjusts (reduces) the interval between asks of the similar questions from a layer in the retrieval practice schedule, so that the learner is most likely to remember and be able to apply the recent feedback.
|
| Expand |
|---|
| title | Avoiding pitfall: learners often subvert the pre access process |
|---|
|
A traditional pre assess process which asks all the learners the same questions at the same time - whether they answer within a test, selecting from multiple choice options or using mini white boards - makes many learners uncomfortable. “Our cohort” are often especially uncomfortable, so they often undermine its effectiveness by quietly not engaging, copying or otherwise avoiding answering, perhaps by claiming they know everything or nothing or perhaps by asking - when will I use this in life? That’s why timely practice ensures that pre assess and retrieval practice are "low stakes". Learners, over time, learn their assessment is formative (to help their learning) not summative (to judge them against a standard). We need to teach our cohort - who no matter what the standard was, have learned that they won’t meet that standard - that when they use timely practice we are in the business of assessing them to help them learn better. (We probably won’t be able to avoid summative testing entirely.) it may take some time for learners to begin to feel comfortable, what the teacher can do to help is keep sticking with the message: “I’m finding out what will be easiest and most valuable to teach next” and “I’m finding out how well you can recall recent learning, so that you can recall it more easily in the future” and encouraging learners to “remember to use a question mark, if you need to”.
|
| Expand |
|---|
| title | Why doesn’t timely practice ask each learner a question from every layer in every topic? |
|---|
|
We don’t ask each learner a question from every layer in every topic because it would take too long: we initially rule out some key layers that we consider too easy or too hard (based on the Level for Learner the teacher inputs into the app) and because it would be too traumatic for learners. Hence the teacher can say to the class “Everybody will get some too easy, some OK and some too hard questions in their learned and remembered section, this is to help the app find out for the teacher, what to teach you soon.”
Instead we ask questions on a few key layers of a topic, and any that are answered 100% correctly we ask an additional verify question.
This gives us a broad brush stroke picture of the learner’s skills and learning gaps. We then ask responsively questions from easier or harder key layers, and any that are answered 100% correctly we ask an additional verify question, then we sometimes ask questions from some scaffold layers and any that are answered 100% correctly we ask an additional verify question.
This trial and improvement process means we can be fairly confident we are finding each learners' firm learning foundations from each topic. |
| Expand |
|---|
| title | Rarely/never teach too much nor too hard |
|---|
|
It’s easy to teach too much and/or too hard and if we do the learners will forget the new teaching and will inevitably become progressively more despondent about their ability to learn, the retrieval practice system that timely practice uses would require more feedback capacity than the teacher has available and so only learning that the learner requires no feedback for would be retained 😢
The pre assess process is set up so that the teacher can create and set an assessment assignment for one whole lesson or shorter assessment assignments to be completed in say 10 to 15 minutes of a fortnight of consecutive lessons,
After this, the timely practice app can show the teacher firm foundations for the first few topics. The app is intended to hold the teacher back from teaching too much and/or too hard, meaning the learners generally retain their learning until the next lesson, and when this happens regularly learners become progressively more motivated, very little feedback is required to ensure the retrieval practice system that timely practice uses ensures learners retain new learning for the long term
|
(2) Assess Pre-Assess Only Assignments
| Expand |
|---|
| title | Read about the process of creating the first set of Warmup (pre-assess only) assignments |
|---|
|
Preparing for lessons requiring a warm up assignment, Create a P&P (planning and preparation) session - the type will be Warmup and the date will be the date of the lesson when timely practice pre assess will happen Open that P&P session and tap Continue to move past the Assess t.p. section (this is the first assignment, so there are none to assess yet). Change the nominal number of questions (if you want a longer or shorter assignment). Leave both retrieval practice and pre assess assignments ticked (this is good practice for 2 reasons which are explained later) Tap the Create button, tap Continue Select the type of Download you prefer: duplex concatenated (all learners assignments printed double sided with additional blank pages to easily separate them), simplex (single sided) or each learner separately. Click the appropriate green arrow(s). The assignments will be found in your downloads folder.
|
The 3 pre assess assessment options in pre assess are tick, bell and reset
tick | bell | reset |
|---|
 Image Added |  Image Added |  Image Added |
tick is for a fully independent and accurate answer
BELL, stands for BEst Learned Later. It is used for an almost (but not fully correct answer) or for a one part correct answer or for a question not answered (but the teacher assumes that the learner has had a chance to read and try to answer the question).
reset is for when the teacher assumes the learner hasn’t had a chance to answer the question e.g. the last few questions of the assignment are all left blank (the learner seems to have run out of time) or e.g. a blank answer line, when a short question is sandwiched between longer questions (the teacher thinks that the learner may not have seen the question). This assessment option can also be used if the teacher can’t quite read the learners answer e.g. a 3 has been changed to a 5 (the pre assess process is not a test, the purpose is to find out what the learner already knows, later the teacher can train the learner in good exam skills).
Assess the first pre-assess assignment
...
Read about the process, rather than watch the mp4s
| Expand |
|---|
| title | Read about the process |
|---|
|
Preparing for lessons requiring a warm up assignment, Create a P&P (planning and preparation) session - the type will be Warmup and the date will be the date of the lesson when timely practice pre assess will happen Open that P&P session and tap Continue to move past the Assess t.p. section (this is the first assignment, so there are none to assess yet). Change the nominal number of questions (if you want a longer or shorter assignment). Leave both retrieval practice and pre assess assignments ticked (this is good practice for 2 reasons which are explained later) Tap the Create button, tap Continue Select the type of Download you prefer: duplex concatenated (all learners assignments printed double sided with additional blank pages to easily separate them), simplex (single sided) or each learner separately. Click the appropriate green arrow(s). The assignments will be found in your downloads folder.
|
(3) Absent Learner List - ensures a maximum of 1 open assignment per learner
For the second and subsequent Warmup assignments, we need to make sure that each learner has a maximum of one open assignment. In this mp4, we see how learner F (who was absent last lesson), didn’t have their assignment assessed (because they didn’t complete it, because they were absent) and the teacher makes a list/remembers who the absent learners were and unticks these learners' names, before pressing the Create button in the Create t.p. section of the planning and preparation session.
...
| Expand |
|---|
| title | Read about the process rather than watching the mp4s |
|---|
|
Preparing for lessons requiring a warm up assignment, Create a P&P (planning and preparation) session - the type will be warmup and the date will be the date of the lesson when timely practice pre assess will happen Open that P&P session and go to the Assess t.p. section and select a learners name Assess each question of the learners' completed/partially completed assignment using tick: fully correct BELL (BEst Learned Later): almost any other attempt e.g. partially correct, incorrect, missed out, learner wrote a ? etc reset: learner ran out of time or you think the learner didn’t see the question (e.g, small question sandwiched between larger questions, you just can’t read the learners handwriting e.g. they orginally wrote 5 they’ve overwritten with 3, or was it the other way around?) once all the questions in a learners assignment are assessed, tap the floppy disk symbol and assess the next learner’s assignment
Once all completed/partially completed assignments have been assessed, make a note of who has an outstanding assignment (and check you have the paper copy) and tap the Continue button If you need to, change any learners Pace for Practice, (this will ensure most students complete their assignment within a minute or two of each other). Each learner’s Pace for Practice slider, is found within each learner’s data within the Classes/Learners' tab. Then, return to the Planning & Preparation Tab, for the next lesson and return to the Create t.p. section. Change the nominal number of questions (if you want a longer or shorter assignment), and tap to un tick the names of the learners who have an outstanding assignment. Leave both retrieval practice and pre assess assignments ticked (this is good practice for 2 reasons which are explained later) Tap the Create button, tap Continue Select the type of Download you prefer: duplex concatenated (all learners assignments printed double sided with additional blank pages to easily separate them), simplex (single sided) or each learner separately. Click the appropriate green arrow(s). The assignments will be found in your downloads folder.
|
(4) Assess last lesson’s teaching + Decide between FOA, FOB and BELL:
Once we begin to use timely practice for teaching, we will create a tpTeach style of Planning and Preparation session.
Using the Absent Learner List in Edit Taught
If a learner has been absent from a lesson, then we don’t want to add the layer we wanted them to teach them, into their retrieval practice (as we know they haven’t learned it). So using the Absent Learner List, tab the yellow squares of the layers of the absent learners. We aren’t going to judge whether we think the learner learned what was taught last lesson (as it won’t be accurate), we are going to leave the timely practice app to find out whether the learner has retained until the next lesson, what they were taught last lesson.
Principles to Decide between FOA, FOB and BELL
In a nutshell the question is
Will the learner, in the next week or two, “need too much help” to embed this learning?
and the answer to this question depends on the learning context.
FOA: feedback on attempt, the learner has made an attempt and the teacher thinks it’s worthwhile to give feedback, the symbol is a speech bubble with writing inside,
FOB: feedback on blank, the learner hasn’t made an attempt but the teacher still thinks it’s worthwhile to give feedback, the symbol is an empty speech bubble,
BELL: BEst Learned Later, the teacher and/or the learner think that feedback isn’t working, the symbol is a bell.
| Expand |
|---|
| title | Considerations when thinking "perhaps I should BELL this layer?" |
|---|
|
The decision between feedback (FOB or FOA) and BELL (BEst Learned Later) can be complex, as we are deciding on the best use of the teacher’s and learner’s lesson time. If the learner made errors/didn’t answer a recently taught topic, usually we will use the correct feedback symbol. However if the learner made errors/didn’t answer a topic they’ve had feedback on a few times already, then it’s useful to decide “shall I BELL this layer?” The question isn’t about just whether the teacher and learner can use the feedback process to “get the learner to be able to ask similar questions in the layer” but also about balancing the cost in lesson time allocated and the cost to the learner’s limited supply of motivation. Questions which might help decide are does the learner still have some motivation for the feedback process? given the number of learners in the class/group - and how much time the teacher normally has to give each learner feedback - is giving more feedback on this layer sensible? e.g. a tutor with fewer learners in their group, might give choose feedback whereas a teacher with a larger class might choose BELL, thinking about the next fortnight, will there be enough time to embed the feedback into the learner’s long term memory? e.g. if there are only 2 days before a 2 week holiday, then the learner is unlikely to retain the benefits of feedback but if there is more than 2 weeks before the next holiday, the teacher might want to try to give a bit more feedback.
|
| Expand |
|---|
| title | sometimes BELL (BEst Learned Later) may result in more embedded learning than feedback |
|---|
|
if the feedback on a layer isn’t going well, the teacher and learner are likely to remember, perhaps with a slightly sick feeling in the stomach, how the previous feedback-dialogue attempts went. They should both be honest about whether the feedback is working, in a case like this, deciding on BELL will open up more time and more motivation for other feedback. the more feedback assessments per assignment, the less likely each is to result in embedded learning - usually 3 feedbacks per assignment is enough, so if a layer which has already had several feedback dialogues is the only layer needing feedback perhaps continue, but if there are 2 or 3 other layers needing feedback perhaps BELL the layer which has had several feedback’s already.
|
(5) Teachers Assess (but don’t Mark) the timely practice assignments
| Expand |
|---|
| title | Assessment is about improving future learning, whereas marking may have different purposes |
|---|
|
The purpose of marking may be thought to be To make sure there are no remaining written errors on the page: the reason might be that then the learner can revise from their exercise books. This is not a good use of time, because we want learning to be improved soon, rather than hope learners will return and revise at a later date. To give hints or clues or model answers to help the learner. However the teacher must guess what the learner was thinking/ not thinking when the learner made their error(s). This is not a good use of time because often the teacher will guess wrong, or the learner won’t read, or understand if they even read what the teacher has written. The teacher gives value judgements in the hope of changing learners effort/motivation in lessons. This is not a good use of time, because value judgements, even positive ones, may decrease rather than increase the likelihood of learners learning from suggestions and the learners motivation. A repetition of all or part of the original teaching. This is not a good use of time, because if the teaching of the lesson, didn’t lead to embedded learning, the teachers explanation - which is necessarily less detailed and can’t involve assessment for learning, as the lesson could - is less likely to lead to embedded learning.
Although the desired marking purpose may sometimes be achieved, the probability that any of these purposes will effectively embedding learning is low, and the cost to the teacher is high. |
If the teacher finds, when assessing an assignment, that a learner is unable to independently and accurately answer a question, and the teacher decides to give feedback the next lesson, then this feedback-dialogue in the classroom has multiple benefits over marking (These are described in Top Tip 2).
The assessment outcome for each question should be communicated to the learner and the app. The most efficient way to do this is
| Expand |
|---|
| title | Top Tip 1: write assessment outcome, with writing hand, on the pdf + tap, with non-writing hand, on the correct assessment outcome in the app |
|---|
|
This saves the teacher a little time for each question. |
| Expand |
|---|
| title | Top Tip 2: only write the assessment outcome: don't give value judgements, don't give hints, don't correct working, don't try and find where or why errors were made |
|---|
|
Teacher’s may find suppressing their urge/habits to write more a little difficult at first - but stick with it - not only will it reduce the teacher work load in non-directed time, it will also make retrieval practice more effective at embedding learning. The only exception to this rule, is for example, if the learner makes an error in a complex multistage word problem, the teacher might want to write down the numerical answer (on the learner's page, or on the teacher’s lesson plan page) to save the teacher having to work out the answer to check the learner's correction. If the assessment outcome is a tick, BELL (BEst Learned Later) or reset: there is no need for feedback in the next lesson. If the assessment outcome is FOA feedback on attempt or FOB feedback on blank: then we expect the learner to try and self correct or get peer to peer help or initiate a feedback-dialogue with the teacher, within the rules of the classroom. By assessing and where necessary, engaging in personalised feedback dialogue in the lesson, we gain multiple advantages: The teacher is replacing the time and effort they would spend on marking, non-directed time, with only the possibility that they may need to spend directed time, inside of lesson time, on feedback. The teacher need not feel guilty, as not marking + giving feedback in the lesson (if necessary), helps the learner learn better. The non-directed time spent by the teacher will be less - today - because this kind of assessment is far quicker than marking, and will be less - in the future - because feedback is far more likely to be successful and so similar questions will be asked less frequently and answered more accurately. The learner has a chance to self correct or self reflect or get peer-to-peer help: so the teacher may not need to spend lesson time giving feedback or failing that, the learner has time to read the question again and will be primed to adjust their chunk or mental schema (Kornell et al 2009) during the feedback-dialogue. With in-lesson feedback-dialogue the learner is far more likely to engage with the teacher than they are likely to engage with the teacher's marking. Without marking by the teacher, the teacher and learner can start the feedback-dialogue from a common place. In feedback-dialogue the teacher has a chance to learn about learner's past thinking and/or influence the learner's future thinking, more effectively than with marking. Feedback-dialogue makes excellent, non threatening, feedback for the teacher on fine detail ways to improve future teaching. It allows the teacher to gain decades worth of high quality teaching experience within a much shorter span of time. Feedback-dialogue provides an opportunity for the teacher to help the learner to better deal with the emotions brought up by errors e.g. to move on from self-criticism or making excuses or blaming others and to help the learner reflect about their question reading or process or problem solving skills i.e. help the learner grow a growth mindset. Feedback-dialogue is likely to increase the learner's motivation whereas marking is likely to decrease it. Sometimes during feedback-dialogue the teacher and/or learner will realise that the layer is BELL (BEst Learned Later). This is a productive use of the teachers and learner's time - because now they can stop putting their time and effort into making this layer stick and instead put their efforts in to embedding other layers which are far more likely to stick.
| Info |
|---|
After all we can’t expect to go from a place where most teaching is forgotten to a place where absolutely all teaching becomes embedded learning. Surely a process where almost all teaching becomes embedded learning should satisfy us - there is no shortage of other layers to teach - before returning, to see if this layer will stick in a few months time. |
The main disadvantage (such as it is) is that there is very little in the way of an easy to follow paper trail of the efforts the teacher has put in. There is the record that feedback dialogue has been requested, and possibly the corrected workings by the learner. In future assignments we usually see the learner being able to answer correctly similar questions and there is the record of the subsequent progress on the layer within the timely practice app. If the teacher must be checked up on, then the “checker-upper” must do the work to find out if the feedback is successful or not. This counterposes with a marking trail - where the teacher is doing the work of demonstrating that they have tried - but cannot possibly demonstrate that they have been successful - in enabling the learner to embed the learning. |
| Expand |
|---|
| title | Top Tip 3: if the assessment outcome is FOB, position the asterisk** so that the learner can see it, but preserve almost all of the answer space. |
|---|
|
Not giving hints etc will pay dividends, as the answer space can be used to answer the question by the learner and teacher to model answering if the learner gets peer-to-peer help or feedback-dialogue with the teacher by the learner if the learner can actually do the question - perhaps they just didn’t see it, they got distracted, they were having “an off-day” etc
** or other symbol used to show that the answer is not fully correct but that the teacher wishes the learner to try and self correct/get peer help/get feedback |
| Expand |
|---|
| title | Top Tip 4: use the answers within timely practice app |
|---|
|
Even though many questions will take the teacher "next to no time" to work out the correct answer, the teacher should use the answers provided by the app, because the small bits of extra time to read the question and work out the correct answer add up, it distracts the teacher from the assessment task - i.e. how well is this learner able to apply their learning - it is harder, and therefore takes fractionally longer, for the teacher to switch back and forth between doing maths and assessing maths.
|
(6) Assess t.p. Retrieval Practice part of the assignment
The 3 pre assess and 5 retrieval practice assessment options
| tick | FOA | FOB | BELL | reset |
|---|
|  Image Added | Speech bubble with writing inside | Empty speech bubble. |  Image Added |  Image Added |
pre assess | independent and accurate answer | N/A | N/A | not fully accurate | The learner ran out of time or seems to have not seen the question. If this happens semi-regularly, reduce the Pace for Practice |
retrieval practice | independent and accurate answer | question attempted, and feedback might work | question not answered, but feedback might work | feedback isn’t working |
(7) Get the most from feedback by remembering it's better called feedback-dialogue
| Info |
|---|
Every time we help a learner fix a chunk in long term memory - when they otherwise would be left having learned no chunk or an imperfect chunk - we are ensuring the learner will make more progress when they otherwise would. |
Feedback should be done after some teaching has become embedded learning, so sometimes reteaching - especially after a long gap between lessons - will be more efficient than giving feedback. If the long gap between lessons can be foreseen, consider planning a cooldown.
Feedback can be more properly thought of as feedback-dialogue, a dialogue between the learner and the teacher - it should be more personalised than reteaching e.g.
help the learner add the bit they have forgotten of a skill or process, to the bit that they have remembered of the skill or process,
if accuracy is an issue - help the learner to check though their workings out - with the ultimate goal that the learner begins to be able to do this for themselves,
adapt the learner's past thinking to influence the learner's future thinking,
use the opportunity to train the learner to figure out what strategy or knowledge will ensure the learner can solve similar problems in the future,
help the learner to better deal with the emotions brought up by errors e.g. to move on from self-criticism or making excuses or blaming others and instead help the learner reflect about their question reading or process or problem solving skills, without fear of feeling a failure or a fool,
sometimes feedback gives the teacher and or learner the opportunity to realise that the layer is best learned later.
Additionally, the process of feedback-dialogue makes excellent, non threatening, feedback for the teacher on the fine details for future teaching.
| Expand |
|---|
| title | Types of chunk problems that feedback can overcome |
|---|
|
The learner has built an incomplete chunk in long-term memory - despite timely practice layers being small and therefore easier to learn, sometimes the learner will need more support to build a chunk - the teacher, via feedback-dialogue, should work with the learner to find what is missing and help the learner fix it. See chunk-basedTheory(4) , for more about this. The learner has not replaced/adapted an incorrect chunk built some time ago, with a new/adapted chunk in long-term memory. This is different from 1. in that the chunk to do the old incorrect method hasn’t been overwritten, despite perhaps a new chunk being built in the lesson. So here we are working on changing the trigger i.e. the learner choosing the new correct chunk, rather than the old incorrect one. The best way to do this, is to offer a reason why the old incorrect one is incorrect or inefficient which chimes with the learner’s understanding. The learner is still reliant on some of the “unacknowledged scaffolding'' of the lesson e.g. placement of workings out on the page or use of a diagram etc. See chunk-basedTheory(4) for more about this. The learner has misread the question or poorly applied their numeracy skills when answering the question. This is likely to be due to working memory overload - all learners, even the most able A level maths learners experience it. As they are learning and working through something new and hard, they are unable to accurately apply skills which are usually easy for them. I sometimes describe this effect to students as their brain isn’t very good at easy thinking and hard thinking at the same time. The best we can offer learners as they practise, is that they can look through their workings out for accuracy periodically. Sometimes I suggest they write “check for accuracy” on the answer line, as an aide memoire, for when they think they have solved the problem. We need to encourage learners to realise that making “silly mistakes” is often a sign of hard learning going on, not a sign that “they can’t even do the easy maths”.
|
Examples of feedback-dialogue are given within the questions for this layer (ask if you would like this training).
(8) Feedback-dialogue is best after a good night's sleep
Contrary to our expectations, feedback is better given after one sleep, than directly after an error. It seems that if we give feedback on the day of the error, we may not be as effectively triggering reconsolidation - see chunk-basedTheory(2) - that is we are not as effectively triggering the brain to change chunks in long-term memory. On the other hand, if we leave feedback for too many days, then feedback is not as effective as it could be, because forgetting has further deteriorated the learning our feedback is trying to fix.
| Expand |
|---|
| title | timely practice offers feedback the next maths lesson - which is close to ideal |
|---|
|
I wish I could say that is why timely practice expects the teacher to assess assignments between one lesson and the next, so that feedback is delayed by at least one sleep. However the main reasons for this fundamental decision are: to reduce the need for learners to having one computing device per learner and the need for reliable access to wifi in every lesson, to get learners to write by hand their workings and diagrams, just as they must in a test (which may also be more effective at embedding learning).
|