Research

How Can Detailed Feedback From Robots Hinder Human Learning?

how-can-detailed-feedback-from-robots-hinder-human-learning

More recently, both the concept of AI tutors and humanoid robots appear destined to be found alongside teachers in the classroom, supervisors on a production floor, and helpers at home; we face an ever-greater decision about the extent to which personalised, in-depth learning experiences benefit rather than hinder their trainees, and for now the answer appears to depend not only on the trainee, but on the feedback being delivered.

The study published in Neuroscience News investigated how adults reacted to different types of robot failure feedback during the completion of a tricky puzzle in a second language as a way to assess one facet of a crucial conflict between immediate and ultimate performance: the level and personalised nature of automated failure feedback. 

How feedback impacts learners in terms of efficiency & task load

The main idea that the research is focused on is how cognition and emotion work in real-time during technology-based learning following failure feedback. Essentially, feedback upon making mistakes can be highly conducive to learning while also increasing mental load at a point when the learner is already potentially overloaded. Cognitive load describes the rate of information processing that the brain can handle: when feedback messages are very wordy and descriptive, they may overload the learner’s processing capacity. 

The hypothesis, however, on which this research is based, is that feedback must be adapted to not only the cognitive capabilities of the learner but also the learner’s emotional state at that moment, such as their level of boredom, to have maximum learning benefit. This study addresses one main psychological tenet: that more detailed, personalised information does not automatically equate to more helpful information. 

Before outlining the experiment, it is useful to specify the kinds of feedback involved. Cognitive feedback can be directly informative about the actions to perform in order to learn a given task (such as showing where an object belongs). Metacognitive feedback consists of asking learners to think back to strategies or choices that they have made previously. Personalised feedback indicates error messages relate to a specific prior mistake made by the learner (e.g., trying to place the object at the wrong spot for the umpteenth time), while content-generic feedback provides more general hints not contingent upon the exact history of past mistakes made by the learner. 

Experimental design

In order to examine such processes, Swahili speaker-directed robots and household objects, such as bottles, cups, and books, were used. Observers stood on the floor and determined the possible positions of each target above them (i.e., “on the table” or “on the chair”) based solely on instructions given in Swahili, an unfamiliar language for the participants. Such a situation allows us to consider placement as involving two concomitant tasks: at once, spatial reasoning and a gradual construction of linguistic meaning. After participants moved a target to a chosen position, researchers provided either feedback indicating whether the placement was successful or additional cognitive explanations (i.e., explaining a property of the location) or metacognitive feedback in some experimental conditions.

In total, 90 participants (18–59 years) with an equal distribution of genders participated in the experiment by Helene Ackermann (first author), Anna L. Lange, Hanna Dumont, Verena V. Hafner, and Rebecca Lazarides in the Science of Intelligence (SCIoI) Cluster of Excellence in Berlin. Participants were randomly assigned to one of three conditions of feedback: 

  • The condition-featured guidance featured content-generic feedback with fixed frequency that did not consider prior task performance. 
  • In the basic-adaptive condition, the feedback frequency was adapted to recent performance and enjoyment scores; feedback content, however, was content-generic. 
  • In a person-adapted condition in which feedback frequency and content had been adjusted to each of the learners, the feedback was about own errors and previous steps. 

Researchers used a nested design involving a series of trials organised into daily blocks and individual learners to estimate a three-level generalised path model. The model examined whether failure feedback affected the probability of success on the subsequent trial and whether this relationship was moderated by feedback design and learner characteristics, such as general aptitude and state boredom.

The correlation between failure feedback and correct response

Based on their analyses, the researchers noted, “Cognitive or metacognitive failure-contingent feedback usually increased the subsequent probability of a correct answer over unfeedbacked answers,” and the participants’ successful performance on the follow-up attempt would have resulted. Therefore, they concluded that “error-contingent guidance is useful for robotic-assisted learning,” but “the frequency adaptation feedback, as implemented here, did not benefit participants significantly more than fixed guidance” in terms of immediate improvement. “Adapting the density of the information, but not its content,” was insufficient.

In contrast, personalised feedback was more complex. Overall, these messages tended to be counterproductive for task-specific advice in the next response; alternatively, receiving extensive advice about past performance immediately after failing hurt learners’ ability to take it into immediate consideration. This seems most likely because of over-extension; personalised feedback generally took more time to decipher due to the lengthy and individual history. However, overall performance was high, with participants performing much better at the end of the task after receiving content-personalised advice than after receiving generic advice. These data exhibit a form of trade-off between long-term learning and the immediate capture of information. 

Similarly, learner characteristics, such as cognitive ability and experience, moderate the effects of feedback on task performance. Cognitive ability moderated the effect of cognitive feedback such that the effect of the latter on task performance attenuated for the more capable learners. Similarly, momentary boredom has also moderated the effect of task-based instruction such that task-based instruction increases in task performance were stronger for those experiencing high momentary boredom. Therefore, the same instructional message affects learners differently based on their individual as well as their present psychological status. 

Author’s interpretation

The researchers highlight that educational software must not adopt unconditional personalisation as an end. Instead, their concern about personalization: The real educational game is dynamic: you need to coordinate personalised support on the one hand with a sensitivity towards the learner’s present situation and a dynamic evaluation of his/her situational cognitive and emotional state on the other hand and to modulate the density and dosage of personalised support on either time scale. Furthermore, they argue that learners should not view personalised feedback as inherently “good” or “bad”. Although it may make it more difficult for learners to focus on the next step immediately after a failure, it can ultimately improve overall task performance when considered across the entire task.

From these individuals’ perspective, “the smartest robotic tutor will not always be the one that spends more effort to explain everything in detail, but the one that will figure out when less is more,” the authors wrote in the paper. “The robot must, according to the study, ‘read’ the situation, which means that robots must not just monitor errors but they have to actively monitor the mental overload of the student,” as well as signals like frustration and boredom. The human authors said that the robots should assist, not replace, human teachers, and the purpose of designing feedback strategies “is to provide enough information at any given time that students do not feel overwhelmed, but not so much as to have them miss the important messages.” 

Conclusion

Collectively, the findings demonstrate that automated feedback from humanoid robot tutors can improve after-error learning, and the level of feedback information and personalisation is critical for whether that help benefits learners at short- or long-term timescales. Personalised, detailed information can hinder learners’ immediate recovery from errors because it increases demands on mental resources when their attention is already occupied by the error. However, over time, this richer information can improve their learning of the task.

In summary, effective smart support in educational technologies depends not only on simply delivering a lot of information but also on delivering just the right amount, at the right time, and adapted to a learner’s cognitive and emotional state, and skill level. As humanoid and robot tutors become more prevalent, learning system designers need to develop systems that can flexibly adjust what they say and when they say it to optimise both immediate support and learners’ long-term performance.

References +
Exit mobile version