Archive

Posts Tagged ‘assessment’

Reading notes on ‘Reflection in higher education learning’

October 29, 2009 7 comments

In his paper, Moon (2001) draws a picture of the conception of reflection in higher education as a tool to a better learning. The main points are also summarised in another article written four years later (Moon, 2005).

Moon begins by searching for a definition of the concept of reflection, suggesting that it is a form of mental processing to achieve some anticipated outcomes, and is applied to relatively complicated or unstructured ideas for which there is no obvious solution. Moon believes that emotions are also part of the process of reflection and may influence the way it is carried on. For Moon the process of reflection can occur only on something when you already have some knowledge about it, therefore reflection from scratch has no sense. I find this point interesting regarding what I notice in my course or in the courses I mentor as referee, when students are asked to think and reflect about a case study at the early beginning of learning about a subject where they have only a few ideas about it. What can we really expect from such a demand?

The article then explores different authors’ approaches to reflection, encompassing Jurgen Habemas, David Kolb, Donald Shon. From this quick browsing I was struck by the description of Kolb’s cycle which was visualised as a spiral later by Cowan (1998)

Moon1

Adapted from Kolb (1984) and Cowan (1998)

The important point I keep from this schematic vision is the principle of ‘cognitive housekeeping’ given to the reflection process explained by Moon which permits integration of old and new raw cognitive material into a new form of knowledge. We can then consider actual knowledge as a big jigsaw puzzle image where each piece has its importance but reveals a higher level of understanding by putting it all together in an way that makes sense.

From his summary of Habermas (1971) I note the three kinds of knowledge he described:

  1. instrumental knowledge – how we understand and control our human environment;
  2. knowledge as the interpretation of human action and behaviour – to better understand the society in which we are living and behaving;
  3. knowledge as a way of acting on the two first forms of knowledge (reflexive knowledge?) – which transforms personal, social and other situations and gives the bases ‘on which we make judgements’.

The most interesting point about Schon (1983) is the separation of reflection into a dichotomy, reflection-on-action and reflection-in-action. If I can clearly see how we can reflect on and what we have done as a process ex-post, I agree with Moon that reflecting during the action helped my thought process.

Mo0n concludes this theoretical presentation claiming that none of the previous authors study the importance of emotion in the the reflection action and he raises the question of whether emotional content is always present in reflection and if so, how it influences its result. He concludes that this important point for him is not answered in the literature so far.

For Moon, we can only see the result of learning but it is difficult to perceive the process. Nevertheless, Moon suggests that learning passes through different stages: Noticing and Making sense which belong to a surface learning approach (Marton (1997)) and Making meaning, Working with meaning and finally Transformative learning which all belong to the deep approach of learning.

Moon and assessment of students reflective work

I agree with Moon that ‘assessment tends to drive student learning’ and that we can force them to follow a deep approach of learning if they see in this a way of succeeding in the learning task. For this to be true, we, practitioners, must believe in the method we use and show the way to students instead of asking them to adhere to our proposition of methodology. For reflection as a learning tool, practitioners must believe in its potential outcomes to foster such behaviour. Moon suggests that positive outcomes of reflexive work are apparent during the period of revision before examination.

As long as reflection is an ‘encouragement for learners to follow their own thinking, to work without a curriculum’, Moon is not in favour of a formal assessment of this material. But on the other hand, he claims as I already pointed out in a previous post, with Crème that whenever we see value in students’ work it has to be assessed.

Moons then suggests that assessment must follow very clear, and maybe new, criteria enabling fair marking. He also suggest that such artefacts could be marked as ‘adequate and passed’ or ‘not yet adequate and not yet passed’, privileging qualitative rather than quantitative marking. I note that Moon’s marking suggestion gives students a chance to improve even after a final marking. I personally agree with this approach which fosters lifelong learning and the fact that learning from a course continues long after the course’s end.

Positive outcomes of reflection in learning

  • Reflection slows down activities
  • Reflection enables learners to develop greater ‘ownership’ of the material of learning and enhances the student’s ‘voice’
  • Foster ‘metacognition’
  • Challenging learners with ill-structured material of learning improves students’ cognitive ability

Comparing these claims with H808, I agree that reflection slows down the process of learning, fostering a deeper approach and a time left to analyse and incorporate new information to create a bigger picture. This is maybe why the course syllabus is short in H808 but the tasks themselves take a long time to be processed by students. I personally already complained in another post about my feeling of a lack of structure in this course, but in the light of Moon’s claims, this is maybe not a lack but a pedagogical way to push students to find their own way toward professionalism and improvement of our  cognitive ability.

Reflexive tools

Comparing Moon’s suggestions of tools to reflect on, most are offered for engagement during H808: learning journal, portfolio, reflection on work experience, reflective exercises. The following proposed tools are, not yet, part of H808: reflection on work-based training, reflection on placement learning, peer and self assessment.

Issues relating to the introduction of reflective activity

Moon points out different issues as follows:

  • students’ ability to reflect is sometimes weak – ‘not all students find reflection easy’;
  • some staff will not understand reflection either
  • cultural issues – ‘some languages do not have a word for reflection’
  • disciplinary issue – ‘the discourse of some subjects are, by nature, more likely to require reflective activity’

References:

Cowan, J. (1998) ‘On Becoming an Innovative University Teacher’, SRHE / OUP, Buckingham.

Habermas, J. (1971) ‘Knowledge and Human Interests’, Heineman, London.

Kolb, D. (1994) ‘Experiential Learning as the Science of Learning and Development’, Prentice Hall, Englewood Cliffs, NJ.

Marton, F., Hounsell, D., Entwistel (1997) ‘The Experience of Learning’, SCottish Academic Press, Edinburgh.

Moon, J. (2001) ‘PDP working paper 4: reflection in higher education learning’ (online), The Higher Education Academy. Available from: http://www.heacademy.ac.uk/assets/York/documents/resources/resourcedatabase/id72_Reflection_in_Higher_Education_Learning.rtf (accessed 29 June 2007).

Moon, J. (2005) ‘Guide for busy academics no. 4: learning through reflection’ (online), The Higher Education Academy. Available from: http://www.heacademy.ac.uk/assets/York/documents/resources/resourcedatabase/id69_guide_for_busy_academics_no4.doc (accessed 2 July 2008).

Schon, D. (1983) ‘The Reflexive Practitioner’, Jossey-Bass, San-Francisco.

Some notes on: “Should student learning journals be assessed?”

Here are some notes I have taken during the reading of Crème (2005) about if, why and how student learning journals should be assessed. Before this reading I had already been thinking for a long time during other courses and even before having begun the MAODE courses, about whether it was a good thing to have a blog or not.

In an essay for H800 (Technology-enhanced learning: practices and debates) I argued that I didn’t use a blog (even if it was requested) because of a lack of time and because I didn’t want to expose my writing (English is not my native language) and knowledge-in-development to everyone. I felt that the content of blog posts could be used by anyone to judge the post’s author and I was not ready to accept this, especially as I had to write in my third language.

With the beginning of H808, the choice is not mine any more and I am forced to post regularly to provide evidences to be enclosed with the different assessments during the course.

After my reading, I begin to see the direction in which the course drives us. During the two other courses I have already taken in the MAODE’s syllabus, we were also students reflecting on their learning, but the learning was more centred on knowledge acquisition, things to know. In H808, so far, I have the feeling that the outcomes are more about becoming self-sufficient and independent. While a student is learning a subject with their peers, a professional must come to a point where he/she can think by him/herself and take the best decision for their clients.

I think, therefore, that this becomes important for tutors in such a course, as well as for students themselves, to see the progress achieved in gaining independence and experience in e-learning. The use of a reflexive journal is thus no longer simply an option but a useful tool for everyone concerned.

The question raised by Crème in his article is whether such a learning process has to be assessed or not. Without responding directly to the question, Crème points out a certain number of issues for both cases:

Where assessment of student learning journals is made

  • students may engage with this form of learning because it is assessed and not because they find it useful;
  • it means that the institution recognises a different way of thinking and then a different way of assessing skills and knowledge acquisition;
  • tutors estimate learning journals are useful and pedagogically important (we assess what we think is important);
  • students may be intimidated by such a big process (i.e., amount of work, difficulties in commenting regularly about what is read, fear of showing their weak points);
  • the risk is that student will be tempted to write journals as if they were a final and definitive product and not as a reflective and formative instrument.

In short Crème states that assessment may have ‘killed off the qualities that the work itself was designed to foster’: to be a more personal tool showing how students’ ideas have changed and to give students ownership of their learning.

The problem of honesty

Writing a journal and making it public or semi-public, exposes the owner whenever their writing is honest and not driven by the desire to present information to please tutors, but to serve their own reflection on the path to knowledge. I agree with Crème that assessing such sensitive work is judging the person more than their abilities. Knowing that learning journals are being assessed will just push students to write about non personal feelings and then turn the reflective blog into another plain essay without heart.

A mid-way solution

As a solution, Crème suggests using learning journals as evidences to quote from during the writing of a formal essay for assessment. These entries are provided as evidences and not assessed for themselves but serve to scaffold the final work  students will present to their tutors for validation. In this way, students are forced to engage with a reflexive tool such as a learning journal and can hold forth on the subject in their own language, fostering questioning more than answering, learning more than knowing.

Regarding the H808’s Course Guide and Assignment Guide, this middle approach to assess an essay (TMA) supported by evidences drawn from our blogs and e-portfolio follows Crème’s suggestion, thus emphasising the importance of the process but keeping formal writing for formal and final assessment.

Reference:
Crème, P. (2005) ‘Should student learning journals be assessed?’, Assessment and Evaluation in Higher Education, vol. 30, no. 3, pp. 287–96. Available from: http://libezproxy.open.ac.uk/login?url=http://dx.doi.org/10.1080/02602930500063850 (accessed 9 October 2009).

Case 11 – Phased online summative assessment in undergraduate accounting

This is the description of the case 11 for activity 2 in course H807

Background and Context

The Business School of Glamorgan’s University ascertains that with the actual system of assessment, the students do not work during the whole term until the assessment period. It results that the pass rates are low and fall off year after year. The University wants to find a way to force the students to work regularly but also to help them to fill their lacks by a system of constant evaluation that should permit to measure student development and identify where additional support is needed. It should also provide feedback on the students’ progress permitting a self-reflective approach of learning.

To stop this ineluctable fall in the pass rates, the decision is taken to change the method of evaluation and to adopt online tools to support the new strategy.

Project description

The new method introduced by the University consists of fractionating the evaluation in several phased online summative assessments. At the commencement of the accounting module the students receive the lecture program with the dates of their online summative assessments and the topic areas to be tested.

A bank of questions (400) was created, separated into topic groupings and comprised multiple-choice, multiple-response, true/false, yes/no and text match questions. Five online assessments were written with computer random chosen questions. Every exam was then tested to ensure the coherence and the equality of difficulty among all.

Each question has a feedback to help the student who would have false answered. A report with feedback and score was given to each student after their exam. The exams took place at different moment during the week due to the number of students and the lack of IT facilities (3 groups of 20 students). To avoid forbidden access to the tests they were password protected with a different password for each session.

Tools in use

The University has chosen to use Question Mark’s Perception product (QMP) a web-based assessment tool which facilitate the creation, the delivery, tracking and management of online assessments, quizzes and surveys.

The tool can be easy integrated into BlackBoard, the LMS used by the Univesity of Glamorgan.

Tangible benefits

For the students
The tangible benefits are twofold : the new method of assessment permits an noticeable improvement in the pass rate. The students feel themselves more framed and supported by having regularly to work and to pass exams. The direct feedback after each exam is also something pointed out as a very good thing.

For the staff
Once the all the creation work done with QMP, the working process to organise new and different assessments is costless. The principle of phased online summative assessment has proven its strength, it will be introduced and adapted for other modules across the University.

Disavantages and Drawbacks

None. A part the initial work to enter the huge number of question, but because the system is durable, the rapport initial costs / time of usage is low.

Conditions of success

  • The project is supported by the CELT (Centre for Excellence in Learning and Teaching)
  • The project costs (mainly to buy QMP) were supported by the faculty and CELT