assess and feedback

A traditional pre assess process which asks all the learners the same questions at the same time - whether they answer within a test, selecting from multiple choice options or using mini white boards - makes many learners uncomfortable. “Our cohort” are often especially uncomfortable, so they often undermine its effectiveness by quietly not engaging, copying or otherwise avoiding answering, perhaps by claiming they know everything or nothing or perhaps by asking - when will I use this in life?

After every lesson where learners completed a warm up assignment,

1. Create a P&P (planning and preparation) session - the type will be warmup and the date will be the date of the lesson when timely practice pre assess will happen

2. Open that P&P session and go to the Assess t.p. section and select a learners name

3. Assess each question of the learners' completed/partially completed assignment using

   • tick = fully correct

   • bell (best learned later) = almost any other attempt e.g. partially correct, incorrect, missed out, written a ? etc

   • reset = learner ran out of time, you think the learner didn’t see the question (e.g, small question sandwiched between larger questions, you just can’t read the learners handwriting e.g. they orginally wrote 5 they’ve overwritten with 3, or was it the other way around?)

   • once all the questions in a learners assignment are assessed, tap the floppy disk symbol and assess the next learner’s assignment

4. Once all completed/partially completed assignments have been assessed, make a note of who has an outstanding assignment (and check you have the paper copy) and tap the Continue button

5. If you need to, change any learners Pace for Practice, (this will ensure most students complete their assignment within a minute or two of each other). Each learner’s Pace for Practice slider, is found within each learner’s data within the Classes/Learners' tab. Then, return to the Planning & Preparation Tab, for the next lesson and return to the Create t.p. section.

6. Change the nominal number of questions (if you want a longer or shorter assignment), and tap to un tick the names of the learners who have an outstanding assignment.

7. Leave both retrieval practice and pre assess assignments ticked (this is good practice for 2 reasons which are explained later)

8. Tap the Create button, tap Continue

9. Select the type of Download you prefer: duplex concatenated (all learners assignments printed double sided with additional blank pages to easily separate them), simplex (single sided) or each learner separately. Click the appropriate green arrow(s).

10. The assignments will be found in your downloads folder.

When the teacher finds each learners firm learning foundations on a topic and uses that data to decide where to start teaching that topic, the teacher can be confident that the learners will retain new teaching because

• the teacher is teaching on firm learning foundations,

• the app schedules a retrieval practice programme for each bite (layer) of new learning and

• that when feedback needs to be given, it has the best chance of success, because the teacher and learner are entering into a feedback-dialogue and the app adjusts (reduces) the interval between asks of the similar questions from a layer in the retrieval practice schedule, so that the learner is most likely to remember and be able to apply the recent feedback.

It’s easy to teach too much and/or too hard and if we do

• the learners will forget the new teaching and will inevitably become progressively more despondent about their ability to learn,

• the retrieval practice system that timely practice uses would require more feedback capacity than the teacher has available and so only learning that the learner requires no feedback for would be retained 😢

The pre assess process is set up so that the teacher can create and set

• an assessment assignment for one whole lesson or

• shorter assessment assignments to be completed in say 10 to 15 minutes of a fortnight of consecutive lessons,

After this, the timely practice app can show the teacher firm foundations for the first few topics.

The app is intended to hold the teacher back from teaching too much and/or too hard, meaning

• the learners generally retain their learning until the next lesson, and when this happens regularly learners become progressively more motivated,

• very little feedback is required to ensure the retrieval practice system that timely practice uses ensures learners retain new learning for the long term

Of course teacher’s don’t need to use timely practice to plan teaching, but if the teacher wants to use the power of the timely practice app to

• ensure they only ever teach on firm learning foundations and

• automatically schedule personalised retrieval practice on the learning of the lesson and

• have the app adjust the schedule of retrieval practice when a learner has made an error and had feedback;

then the teacher must wait until the app has finished pre assessing the topic. (There is nothing stopping a teacher teaching a layer to a learner whose pre assess on the topic is incomplete, but the app won’t schedule retrieval practice and the teacher may not be teaching on firm learning foundations.)

A learner’s column in the Plan Teaching topic dashboard

• which has some mid grey and darker blue-grey squares - indicates that pre assess is not complete;

• which only shows light grey squares - indicates that the app does not intend to pre assess the topic (judging all the layers either too easy or too hard);

• which shows a mix of white and light grey squares - indicates the app thinks the learner could learn layers from the topic, but has not learned any yet;

• the (often fuzzy) line between the shades of blue (perhaps mixed with light grey) in the lower part of the column and the white (usually mixed in with light grey) in the upper part of the column is the (often fuzzy) “goldilocks” point where teaching should begin.

The decision between feedback (FOB or FOA) and bell (best learned later) can be complex, as we are deciding on the best use of the teacher’s and learner’s lesson time.
The question isn’t about just whether the teacher and learner can use the feedback process to “get the learner to be able to ask similar questions in the layer” but also about balancing the cost in lesson time allocated and the cost to the learner’s limited supply of motivation.

• does the learner still have some motivation for the feedback process?

• the decision to give feedback or decide a layer is best learned later needs to made within the context

  • e.g. the number of learners in the class/group - a tutor with fewer learners in their group, might give choose feedback whereas a teacher with a larger class might choose best learned later,

  • e.g. the amount of time until the next school holiday - if there is only 2 days before a 2 week holiday, then the learner is unlikely to retain the benefits of feedback - this is issue can be in part dealt with by use of a cool down period,

• if the feedback isn’t going well, the teacher and learner are likely to remember the previous feedback-dialogue and they should both be honest about whether the feedback is working,

• the more feedback assessments per assignment, the less likely each is to result in embedded learning - usually 3 feedbacks per assignment is enough,

The purpose of marking may be thought to be

• To make sure there are no remaining written errors on the page: the reason might be that then the learner can revise from their exercise books. This is not a good use of time, because we want learning to be improved soon, rather than hope learners will return and revise at a later date.

• To give hints or clues or model answers to help the learner. However the teacher must guess what the learner was thinking/ not thinking when the learner made their error(s). This is not a good use of time because often the teacher will guess wrong, or the learner won’t read, or understand if they even read what the teacher has written.

• The teacher gives value judgements in the hope of changing learners effort/motivation in lessons. This is not a good use of time, because value judgements, even positive ones, may decrease rather than increase the likelihood of learners learning from suggestions and the learners motivation.

• A repetition of all or part of the original teaching. This is not a good use of time, because if the teaching of the lesson, didn’t lead to embedded learning, the teachers explanation - which is necessarily less detailed and can’t involve assessment for learning, as the lesson could - is less likely to lead to embedded learning.

Although each outcome of marking may sometimes be effective, the probability that any of these will be effective at embedding learning is low, and the cost to the teacher is high.

This saves the teacher a little time for each question.

Teacher’s may find suppressing their urge/habits to write more a little difficult at first - but stick with it - not only will it reduce the teacher work load in non-directed time, it will also make retrieval practice more effective at embedding learning.

_{The only exception to this rule, is for example, if the learner makes an error in a complex multistage word problem, the teacher might want to write down the numerical answer (on the learner's page, or on the teacher’s lesson plan page) to save the teacher having to work out the answer to check the learner's correction.}

If the assessment outcome is a tick or best learned later or reset: there is no need for feedback in the next lesson.

If the assessment outcome is FOA feedback on attempt or FOB feedback on blank: then we expect the learner to try and self correct or get peer to peer help or initiate a feedback-dialogue with the teacher, within the rules of the classroom.

By assessing and where necessary, engaging in personalised feedback dialogue in the lesson, we gain multiple advantages:

1. The teacher is replacing the time and effort they would spend on marking, non-directed time, with only the possibility that they may need to spend directed time, inside of lesson time, on feedback. The teacher need not feel guilty, as not marking + giving feedback in the lesson (if necessary), helps the learner learn better.

2. The non-directed time spent by the teacher will be less - today - because this kind of assessment is far quicker than marking, and will be less - in the future - because feedback is far more likely to be successful and so similar questions will be asked less frequently and answered more accurately.

3. The learner has a chance to self correct or self reflect or get peer-to-peer help: so the teacher may not need to spend lesson time giving feedback or failing that, the learner has time to read the question again and will be primed to adjust their chunk or mental schema (Kornell et al 2009) during the feedback-dialogue.

4. With in-lesson feedback-dialogue the learner is far more likely to engage with the teacher than they are likely to engage with the teacher's marking.

5. Without marking by the teacher, the teacher and learner can start the feedback-dialogue from a common place.

6. With in lesson feedback-dialogue the teacher has a chance to learn about learner's past thinking and/or influence the learner's future thinking, more effectively than with marking.

7. Feedback-dialogue makes excellent, non threatening, feedback for the teacher on fine detail ways to improve future teaching. It allows the teacher to gain decades worth of high quality teaching experience within a much shorter span of time.

8. Feedback-dialogue provides an opportunity for the teacher to help the learner to better deal with the emotions brought up by errors e.g. to move on from self-criticism or making excuses or blaming others and to help the learner reflect about their question reading or process or problem solving skills i.e. help the learner grow a growth mindset.

9. Feedback-dialogue is likely to increase the learner's motivation whereas marking is likely to decrease it.

10. Sometimes during feedback-dialogue the teacher and/or learner will realise that the layer is BELL best learned later - and this is also a productive use of the teachers and learners time - because now they can stop putting their time and effort into making this layer stick and instead put their efforts in to embedding other layers which are far more likely to stick. After all we can’t expect to go from a place where most teaching is forgotten to a place where absolutely all teaching becomes embedded learning. Surely a process where almost all teaching becomes embedded learning should satisfy us - there is no shortage of other layers to teach - before returning, to see if this layer will stick in a few months time.

The main disadvantage (such as it is) is that there is very little in the way of an easy to follow paper trail of the efforts the teacher has put in. There is the record that feedback dialogue has been requested, and possibly the corrected workings by the learner. In future assignments we usually see the learner being able to answer correctly similar questions and there is the record of the subsequent progress on the layer within the timely practice app. If the teacher must be checked up on, then the “checker-upper” must do the work to find out if the teaching is successful or not. This counterposes with a marking trail - where the teacher is doing the work of demonstrating that they have tried - but cannot possibly demonstrate that they have been successful - in enabling the learner to embed the learning.

Not giving hints etc will pay dividends, as the answer space can be used to answer the question

• by the learner and teacher to model answering if the learner gets peer-to-peer help or feedback-dialogue with the teacher

• by the learner if the learner can actually do the question - perhaps they just didn’t see it, they got distracted, they were having “an off-day” etc

** or other symbol used to show that the answer is not fully correct but that the teacher wishes the learner to try and self correct/get peer help/get feedback

Even though many questions will take the teacher "next to no time" to work out the correct answer, the teacher should use the answers provided by the app, because 

• the small bits of extra time to read the question and work out the correct answer add up,

• it distracts the teacher from the assessment task - i.e. how well is this learner able to apply their learning - it is harder, and therefore takes fractionally longer, for the teacher to switch back and forth between doing maths and assessing maths.