In the frame of an on-going 4-years research project, the Aviation Academy Safety Management Systems (AVAC-SMS) metric for the self-assessment of aviation Safety Management Systems (SMS) was designed based on the Safety Management Manual of the International Civil Aviation Organization and in cooperation with knowledge experts and aviation companies. The particularmetric evaluates three areas, namely (1) the degree of institutionalisation of SMS (design and implementation of processes), (2) the extent of managers’ capability to deliver the SMS processes, and (3) the employees’ perceived effectiveness of the SMS-related deliverables. The metric concludes with a score per area and per SMS component/element assessed, and it is scalable to the size and complexity of each organisation. Results of a survey at 18 aviation companies did not show statistically significant differences in their SMS scores across all three assessment areas but revealed a distance between the area of Institutionalization and the areas of Capability and Effectiveness. Also, differences were detected regarding the scores per SMS component and element within and across companies and assessment areas. The various assessment options offered for the AVAC-SMS metric accommodates the resources each SME and large company can invest in the application of the metric. Even the lowest level of resolution of the SMS metric can trigger companies to investigate further their weaker areas and foster their SMS-related activities. Therefore, the AVAC-SMS metric is deemed useful to organisations that want to self-assess their SMS and proceed to comparisons amongst various functions and levels and/or over time.
In the frame of an on-going 4-years research project, the Aviation Academy Safety Management Systems (AVAC-SMS) metric for the self-assessment of aviation Safety Management Systems (SMS) was designed based on the Safety Management Manual of the International Civil Aviation Organization and in cooperation with knowledge experts and aviation companies. The particularmetric evaluates three areas, namely (1) the degree of institutionalisation of SMS (design and implementation of processes), (2) the extent of managers’ capability to deliver the SMS processes, and (3) the employees’ perceived effectiveness of the SMS-related deliverables. The metric concludes with a score per area and per SMS component/element assessed, and it is scalable to the size and complexity of each organisation. Results of a survey at 18 aviation companies did not show statistically significant differences in their SMS scores across all three assessment areas but revealed a distance between the area of Institutionalization and the areas of Capability and Effectiveness. Also, differences were detected regarding the scores per SMS component and element within and across companies and assessment areas. The various assessment options offered for the AVAC-SMS metric accommodates the resources each SME and large company can invest in the application of the metric. Even the lowest level of resolution of the SMS metric can trigger companies to investigate further their weaker areas and foster their SMS-related activities. Therefore, the AVAC-SMS metric is deemed useful to organisations that want to self-assess their SMS and proceed to comparisons amongst various functions and levels and/or over time.
As part of their SMS, aviation service providers are required to develop and maintain the means to verify the safety performance of their organisation and to validate the effectiveness of safety risk controls. Furthermore, service providers must verify the safety performance of their organisation with reference to the safety performance indicators and safety performance targets of the SMS in support of their organisation’s safety objectives. However, SMEs lack sufficient data to set appropriate safety alerts and targets, or to monitor their performance, and no other objective criteria currently exist to measure the safety of their operations. The Aviation Academy of the Amsterdam University of Applied Sciences therefore took the initiative to develop alternative safety performance metrics. Based on a review of the scientific literature and a survey of existing safety metrics, we proposed several alternative safety metrics. After a review by industry and academia, we developed two alternative metrics into tools to help aviation organisations verify the safety performance of their organisations.The AVAV-SMS tool measures three areas within an organisation’s Safety Management System:• Institutionalisation (design and implementation along with time and internal/external process dependencies).• Capability (the extent to which managers have the capability to implement the SMS).• Effectiveness (the extent to which the SMS deliverables add value to the daily tasks of employees).The tool is scalable to the size and complexity of the organisation, which also makes it useful for small and medium-sized enterprises (SMEs). The AVAS-SCP tool also measures three areas in the organisation’s safety culture prerequisites to foster a positive safety culture:• Organisational plans (whether the company has designed/documented each of the safety cultureprerequisites).• Implementation (the extent to which the prerequisites are realised by the managers/supervisors acrossvarious organisational levels).• Perception (the degree to which frontline employees perceive the effects of managers’ actions relatedto safety culture).We field-tested these tools, demonstrating that they have adequate sensitivity to capture gaps between Work-as-Imagined (WaI) and Work-as-Done (WaD) across organisations. Both tools are therefore useful to organisations that want to self-assess their SMS and safety culture prerequisite levels and proceed to comparisons among various functions and levels and/or over time. Our field testing and observations during the turn-around processes of a regional airline confirm that significant differences exist between WaI and WaD. Although these differences may not automatically be detrimental to safety, gaining insight into them is clearly necessary to manage safety. We conceptually developed safety metrics based on the effectiveness of risk controls. However, these could not be fully field-tested within the scope of this research project. We recommend a continuation of research in this direction. We also explored safety metrics based on the scarcity of resources and system complexity. Again, more research is required here to determine whether these provide viable solutions.
