Research Paper to be presented by NMCI Lecturer at MARSIM 2006
Friday 24 March 2006
Capt. Bill Kavanagh, a Lecturer in Navigation at the National Maritime College of Ireland is to present a paper at the International Conference on Marine Simulation and Ship Manoeuvrability, MARSIM 2006, being held in Maritime Institute Willem Barentsz (MIWB), Terschelling, Netherlands on June 25th – 26th. http://www.marsim2006.com An abstract of the paper is given here.
THE DEVELOPMENT OF SHIP SIMULATION ASSESSMENT TO COMPLEMENT WRITTEN MARITIME EXAMINATIONS IN IRELAND
Bill Kavanagh (National Maritime College of Ireland)
Abstract. This paper is a summary of research which examined the development of ship simulation assessment as a complementary form of assessment to written examinations for competency-based assessment in Coastal Navigation within the Seafarer’s Training, Certification and Watchkeeping Code 1995 (STCW 95 Code) for qualifying stage Officer of the Watch (Navigation) (OOW) students in Ireland in 2005.
Current thinking on assessment principles and the use of simulation for assessment were researched by literature review. A semi-structured questionnaire determined views from Canada (Vancouver and Newfoundland), Sweden, the Netherlands, and Ireland. Simulation assessment is used in various forms outside of Ireland, but in a limited form in Ireland. The STCW Code and examination syllabus from the Maritime Safety Directorate (MSD) of the Department of Transport, Ireland, were used to determine relevant tasks for inclusion in a simulation assessment paper. A simulation task paper was written and a marking system was developed which reflected OOW required task skills, underlying knowledge and proficiency. A purposeful sample of nine qualifying stage OOW students which represented about 50% of the population for 2005, participated in quasi-experiments, using ship simulators, to test the assessment paper. Assessments were marked and compared to the actual results obtained in the students’ official written Coastal Navigation examinations. Marks awarded for the simulation assessment averaged 62%, in contrast to the students’ official written examination average mark of 72%.
A comparison of both sets of results resulted in a Spearman Rank Difference correlation +0.73, indicating a strong relationship. Simulation assessment paper task analysis indicated easy and difficult tasks for review. Participants’ views, obtained using an etic reaction questionnaire, showed that the students enjoyed simulation and found it realistic; believed it related well to tasks during training; and found some task completion difficult. After the written examination and simulation assessment process, an emic focus group interview with the participants confirmed the etic view and resulted in valuable insights and suggestions for improvements. Students felt that simulation assessment was more difficult than the appropriate written examinations.
The research concluded that simulation assessment is a suitable means for determining competency, as required by STCW 95, in contrast to the existing written examination. Simulation assessment which is relevant to International standards is valid, reliable, practical and authentic. It can be developed and used successfully to complement existing professional written examinations for OOW students in Coastal Navigation, using continuous assessment. This research is the first of its kind in Ireland, and lays the foundation for the future expansion and progression of assessment systems in Irish maritime education and training.
