Re-audit of the Quality Assurance System of the Seinäjoki University of Applied Sciences

Transkriptio

1 Inkeri Papp Eric Lindesjöö Riikka Töytäri Hannele Seppälä Re-audit of the Quality Assurance System of the Seinäjoki University of Applied Sciences PUBLICATIONS OF THE FINNISH HIGHER EDUCATION EVALUATION COUNCIL :

2 ISBN (paperbound) ISBN (pdf) ISSN Publisher: The Finnish Higher Education Evaluation Council Cover: Juha Ilonen Layout: Pikseri Julkaisupalvelut Esa Print Oy Tampere 2009

3 Preface The Finnish Higher Education Evaluation Council (FINHEEC) will have audited the quality assurance systems of all Finnish higher education institutions by The main aim of the audits is to support the higher education institutions in the development of their quality assurance systems to correspond to the principles of European quality assurance 1 and to demonstrate that a competent and consistent national higher education level quality assurance is in place in Finland. Consequently, one of the aims of the audits is also to positively affect the competitiveness of Finnish higher education institutions in global education markets. The national aim is to collect and disseminate good practices in quality assurance, promote their spread within higher education and thus to develop higher education as a whole. The point of departure of quality assurance audits is enhancement-led evaluation, which has strong traditions in the Finnish evaluation practice and which the higher education institutions have found to be supportive of their own activities and autonomy. The method is based on trust in the higher education institution s own responsibility for the quality of its activities. The higher education institution decides on the form its quality assurance system takes, while the audit evaluates its appropriateness, comprehensiveness, functionality and impact. The development phase of the audit model took place in In November 2007, FINHEEC published the new edition of the audit manual 2, which defines the aims, objects, methods, criteria and sanctions of the audits. The manual follows the general principles and methods of the previous manual, but certain focusing and specifying changes were made to it on the basis of feedback from the higher education institutions and auditors as well as the experiences of the Evaluation Council. Feedback will be collected also in the future from all audit participants and, in 2009, a first follow-up and development seminar for higher education institutions audited approximately three years earlier will be organised. During 2009, the development of the secondround evaluation methodology will also commence. 1 Standards and Guidelines for Quality Assurance in the European Higher Education Area. European Association for Quality Assurance in Higher Education. Helsinki: Multiprint. ( / 2 Audits of Quality Assurance Systems of Finnish Higher Education Institutions. Audit Manual for Finnish Higher Education Evaluation Council publications 10:2007.

4 According to the feedback received from the higher education institutions, the audits have clearly expedited the systematic development of quality assurance systems and methods. It appears that quality assurance has both provided tools for the internal administration of the higher education institutions and steered the institutions to develop their activities as a whole. It may well be said that the audit processes and public reporting on the higher education institutions systems have increased and expanded the discussion on quality and interaction between higher education institutions and their stakeholders. Experiences of the significance of re-audits have also been gained. They provide a directed development impulse and a timetable, and promote the development of the quality assurance system. It is also worthwhile viewing the re-audit as an opportunity to achieve a particularly good level of quality assurance instead of a mediocre one. The method is similar to the actual audits. The difference between the first round and re-audit reporting is that no new development recommendations will be given. The responsibility for utilising and applying the audit information rests with the higher education institution and audits already performed demonstrate that the method promotes the fulfilment of this responsibility. On behalf of the Finnish Higher Education Evaluation Council, I wish to thank the Seinäjoki University of Applied Sciences for participating in the re-audit. I also wish to thank the re-audit group for its expert work and commitment. Riitta Pyykkö, Professor Chair of the Finnish Higher Education Evaluation Council, FINHEEC

5 Contents Re-audit group Re-audit objectives and criteria Re-audit process. Negotiation between the higher education institution and FINHEEC. Re-audit agreement. Re-audit material. Re-audit visit. Structure and compilation of the re-audit report SeAMK and its Quality Assurance System. Organisation structure and administration. Vision, mission and strategy. SeAMK s Quality Assurance System.... Results Objectives, overall structure and internal coherence of the quality assurance system Documentation, including the formulation of a quality policy and the definition of procedures, actors and responsibilities Final conclusions Audit group s overall assessment of the quality assurance system of SEAMK FINHEEC s decision Appendices : Table of audit criteria used in the SeAMK re-audit : Programme of the re-audit visit

6

7 Re-audit group Chair Inkeri Papp, Finland Inkeri Papp, Counsellor of Education, was formerly Head of the School of Health and Social Care at Jyväskylä University of Applied Sciences. In this position she also acted as a vice-rector of the Jyväskylä University of Applied Sciences. Mrs. Papp participated in FINHEEC s international evaluation of the postgraduate polytechnic experiment in Finland as a member of the evaluation team in She also participated in the evaluation of the quality work of the South Karelia Polytechnic 3 in 2000, as well in the evaluations of the professional courses accredited by FINHEEC. Inkeri Papp acted as the chairperson of the audit group of SeAMK s quality assurance system in Members Dr. Eric Lindesjöö, Sweden Dr. Eric Lindesjöö is a Project Manager at the Swedish National Agency for Higher Education (Högskoleverket) and from the beginning of 2001 has primarily been responsible for the evaluation of subjects and programmes within the natural sciences and technology, such as the evaluation of engineering education in Sweden. Since 2008, he has been the coordinator of the third round of quality audits of Swedish universities and university colleges. He recently took part in an international study that compared the systematic quality work at four Nordic higher education institutions. The study was conducted by the Nordic Quality Assurance Network in Higher Education (NOQA). Eric Lindesjöö participated in the audit of the SeAMK quality assurance system in Riikka Töytäri, Finland Riikka Töytäri, Bachelor of Environmental Management, graduated from Tampere University of Applied Sciences (TAMK) in She was a member of the auditing group of Lahti University of Applied Science s quality assurance system audit in 2006 and was also a member of the Board of 3 From the beginning of 2009 Saimaa University of Applied Sciences.

8 SAMOK in 2004, her expertise being in the field of careers and recruitment. She was an active member of TAMK s student union while at university. She has been also the TAMK representative in FINHEEC s project entitled Student in evaluation of education. Project Manager Hannele Seppälä Hannele Seppälä is a Project Manager in the FINHEEC secretariat.

9 Re-audit objectives and criteria The Audit Manual for states that, based on the audit group s proposal, the Finnish Higher Education Evaluation Council (FINHEEC) will decide whether the higher education institution s (HEI) quality assurance system (QAS) passes the audit or whether a re-audit is needed. FINHEEC will decide if further measures should be taken by the HEI and will impose a reaudit if the HEI s QAS or the quality assurance of the basic mission has major shortcomings in relation to the audit criteria. The re-audit will be conducted two years after the audit proper. If FINHEEC decides that a re-audit is needed, the relevant decision will include the essential development needs of the HEI s QAS. These needs will constitute the targets of the re-audit. During the re-audit, the HEI is required to provide clear evidence of progress in the development of its QAS in the areas defined as essential development needs. Furthermore, the stage of development in the target areas of the re-audit must meet the criteria of Developing. Based on the proposal of the audit group and the audit report 5, FIN- HEEC decided in its meeting of 24 October 2006 that the QAS of Seinäjoki University of Applied Sciences (SeAMK) had some major shortcomings and did not meet the audit criteria, hence the development of the QAS calls for action from the HEI, as well as for a re-audit. The decision included the definition of the targets of the re-audit, which were the structure, coherence and documentation of the QAS as a whole. The re-audit of SeAMK is conducted according to the guidelines given in the re-audit manual for the years The criteria that are applied in the evaluation of the SeAMK s QAS are given in Appendix 1. 4 Audits of Quality Assurance Systems of Finnish Higher education Institutions, Audit Manual for , Publications of the Finnish Higher Education Evaluation Council 4:2006 ( 5 Papp, I., Carolan, D., Handal, G., Lidesjöö, E., Marttinen, R., Mustonen, V. & Isoaho, A. Audit of the Quality Assurance System of Seinäjoki Polytechnic. Publications of the Finnish Higher Education Evaluation Council 15: Audits of the Quality Assurance Systems of Higher Education Institutions. Manual for Reaudits Publications of the Finnish Higher Education Evaluation Council 6:2007. (

10 Re-audit process. Negotiation between the higher education institution and FINHEEC According to the re-audit manual, following FINHEEC s decision on the audit result, the HEI will have an opportunity to discuss with FINHEEC the development plan for its QAS. The audit group can also be heard during these discussions. The negotiation between SeAMK, the audit group and FINHEEC was held on 7 December SeAMK presented at that negotiation a plan that included the development measures required in response to the essential development needs included in the FINHEEC decision and the audit report.. Re-audit agreement According to the re-audit procedure, the agreement signed between the HEI and FINHEEC covers the re-audit targets and schedule, the language used in the re-audit, the duration of the re-audit and the consequences, should the HEI not pass the re-audit. The agreement between SeAMK and FINHEEC was signed on 9 June The following re-audit targets were defined in the agreement: 1. Objectives, overall structure and internal coherence of the QAS. 2. Documentation, including the formulation of a quality policy and the definition of procedures, actors and responsibilities.. Re-audit material According to the re-audit manual, the HEI should issue a written report (10 20 pages) on the progress made in the development of its QAS. The HEI s report should also provide written evidence or Internet links to material indicating progress of development of the quality assurance (QA) in essential target areas. Crucial web-based materials must be printed out and integrated into the written material.

11 The SeAMK report was 14 pages in total and presented the key development measures linked to the two auditing targets that had been taken in developing the QAS, as well as ongoing development measures and those planned. The development measures were as follows: 1. Specification of the SeAMK strategy and sub-strategies 2. Development of an internal structure in 2003 and Development of the SeAMK Quality Policy and the responsibility matrix and process descriptions 4. Drafting and revision of the SeAMK Quality Manual 5. Development of a goal realisation evaluation, e.g. management review 6. Systematisation of internal peer reviews 7. Development of quality documentation. The re-audit material also involved 15 different items of proof/evidence concerning the development of the QAS. This proof/evidence included, e.g., the following: The goals of SeAMK s main strategy and sub-strategies and an example of their follow-up Monitoring of SeAMK s quality objectives Student participation in SeAMK s QA Descriptions of the development targets, measures, responsibilities of internal audits Drafting and deploying of rules for different procedures. In addition to the report, SeAMK also provided other materials, by means of which it presented the measures it had taken toward development of the QAS. SeAMK submitted its report and evidence material to FINHEEC 6 weeks before the re-audit visit. In addition to the report and evidence material, the audit group sought the following additional material from SeAMK: 1. A short presentation (table) that stated the: Objectives of the QA QA procedures/evaluation activities Persons in charge Timetable for the procedures 2. A list of SeAMK s working groups. The audit group had also access to SeAMK s QA documentation on SeAMK s intranet (SharePoint workspace). The rector, President Tapio Varmola and quality manager Aira Bragge from SeAMK attended the audit group meeting on 1 December 2008, where they presented the QAS and the audit material of SeAMK.

12 . Re-audit visit The re-audit visit took place in Seinäjoki on 20 January The aim of the visit was to verify the stage of development of the QAS, based on the material obtained. The visit included interviews with the management, teaching and R&D staff members, support service representatives, faculty deans, as well as students. Also, one thematic discussion was conducted on the topic Development, management and use of the quality data. The audit group interviewed 48 persons in total. The programme of the audit visit is given in Appendix 2.. Structure and compilation of the re-audit report The audit group prepared for the audit visit by formulating audit visit questions and by writing up its preliminary findings based on the audit material provided by SeAMK. The audit group had two meetings before the audit visit. After the audit visit, the group continued the reporting process by writing analytical and evaluative texts according to the different areas of responsibility assigned to its members. The project manager was responsible for the first parts of the report, covering the description of SeAMK and its QAS in general. SeAMK was allowed to check the facts in the report before it was published. To guarantee the correctness of the language used, the report was checked by a native English-speaking language expert.

13 SeAMK and its Quality Assurance System. Organisation structure and administration SeAMK is a comprehensive regional institution of higher education that is maintained by the Seinäjoki Joint Municipal Authority for Education. It began its operations on a temporary basis as a polytechnic in 1992, and its functions were established permanently in Currently, there are about 4700 full-time degree students at SeAMK. Approximately 750 students graduate yearly. There is a staff of 403, of which 205 are full-time teachers. In addition, there are 66 employees working in joint services. SeAMK offers instruction in seven areas of education in six localities. Research, development and service activities linked to education are carried out in the faculties concerned. The purpose of the operations performed by SeAMK is to support South Ostrobothnian entrepreneurship-based corporate and service activities requiring a high level of expertise. Integral SeAMK products include: university of applied sciences based degree education: a total of 24 ratified degree programmes (for the year 2009) leading to an applied sciences (UAS) degree, offering starts in 20 programmes in higher UAS degrees: seven degree programmes for 2009 have been ratified for higher UAS degrees, with starts for two programmes in specialised studies in all fields of study teaching in open university applied sciences in all fields of study R&D operations connected with teaching activities in all fields of study services against payment, such as laboratory services, translation, market research services; externally financed education given in all fields of study international on-the-job training international projects foreign-language instruction and internationalisation training international seminars and conferences. 7 This chapter describes the organisation structure, administration and quality assurance system of the SeAMK on the basis of the terminology used in the re-audit material delivered to FINHEEC by SeAMK.

14 The internal administration of the university is coordinated by its Board and President. The university President functions at the same time as the Director of the Seinäjoki Joint Municipal Authority for Education. At the beginning of 2007 the Council of the Joint Board of Directors approved the University of Applied Sciences Development Programme for As a result of the development programme, the organisational structure and the teaching and R&D-related operations are being reduced as of 2008 from nine to five performance units: School of Technology Business School School of Health Care and Social Work School of Agriculture and Forestry School of Culture and Design. Board of the Joint Municipal Authority for Education Council of the Joint Municipal Authority Board of the Joint Municipal Authority Inspection Board SeAMK Administration Board of the University of Applied Sciences President/ Chair of the Joint Municipal Authority SeAMK office Vice President SeAMK Education R&D Director SeAMK Research and Development Services School of Techno logy School of Health Care and Social Work School of Agri culture and Forestry School of Culture and Design Business School Library and Inform ation Services Adminis trative Director Joint Services Figure. Organisation of SeAMK

15 In addition, the Library functions as an independent performance unit. R&D activities embrace both the research and development operations and the service provisions at SeAMK. The internal services are those performed by the university Office as well as the Joint Services (YPA). The organisational structure and administrative system of SeAMK, which came into force on 1 January 2008, are described in the university s organisation chart (Fig. 1). The reporting relationships between the Board, higher management and elected bodies are specified in the Basic Contract, the Joint Municipal Authority Bylaws (perussopimuksessa, kuntayhtymän hallintosäännössä) and the Regulations of the University of Applied Sciences (AMK johtosäännössä). Processes in SeAMK s organisation SeAMK has revised its processes in 2007 (see Fig. 2, p. 16). SeAMK s core processes are instruction/learning and R&D. The instruction/learning process is divided into degree instruction and other extracurricular instruction (non-degree instruction), as well as the internationalisation of instruction in processes. The R&D process is divided into sub-processes: R&D Projects, R&D Services and international R&D cooperation. In addition to instruction/learning and R&D processes, (internal) the process of strategic management and leadership and support service processes are also implemented at SeAMK. SeAMK support processes are: Student affairs administration, Financial administration, Human resources administration, Information administration, Technical services administration and Library and Information services. Process descriptions have been drafted and linked to the Quality Manual. SeAMK s core processes and their sub-processes as well as their support processes have descriptions. A process stakeholder has been assigned to each process. Some processes are also assigned SeAMK process coordination and a process supervisor within each performance unit as well as process documents. SeAMK process descriptions can be found in the SharePoint workspace.

16 Figure. SeAMK process chart. Vision, mission and strategy SeAMK states its mission in the audit material to be 8 : Seinäjoki University of Applied Sciences shall arrange instruction aiming at a high standard of professional expertise in the following fields: cultural and natural sciences, natural resources and the environment, tourist, catering and home economics, social, health and physical education, technology, communications, social sciences, business economics and administration. The University of Applied Sciences shall implement the research and development activities connected with its educational mandate, the goal of which is the advancement of well-being in South Ostrobothnia. SeAMK states its vision for : Seinäjoki University of Applied Sciences shall be a successful, international institution of higher education with an entrepreneurial spirit. 8 Ratified by the Council of the Joint Municipal Authority, June 2003.

17 SeAMK s values are 10 : Professional know-how International networking Active partnership Entrepreneurial spirit. The strategic aims 11 of SeAMK have been divided into four perspectives in accordance with the Balanced Scorecard frame of reference (Fig. 3). STRATEGY MAP Strategic Targets (development plan for 2010 and SWOT): Staff HR1. Improve staff s entrepreneurial way of working. HR2. Develop staff competence in diverse ways. Processes P1. Increase internationality in core processes and select a limited number of partners for extensive cooperation. P2. Fulfil national and international quality control criteria. Resources R1. Double volume of R&D and service operations. R2. Continue centralising functions to improve profitability. Students, customers and partners S1. Modernise course offerings and teaching methods to correspond to future workplace needs. S2. Work in national and international partnership networks. Figure. SeAMK s Strategy Chart 9 Ratified by the Council of the Joint Municipal Authority, May Ratified by the Council of the Joint Municipal Authority, May 2007.

18 . SeAMK s Quality Assurance System SeAMK s Quality Policy is defined as follows: Good quality respective to Seinäjoki UAS is found in its high standard of instruction and R&D activities. We produce highly employable professional experts for the work world. Our staff are skilful and vital. Our external and internal clients as well as our interest groups are satisfied with our operations. Our working methods are specified in the quality manual, which contains the instructions. In our quality assurance, we appreciate the versatile cooperation with our interest groups. We determine the direction of activities by forecasting the future together with our clients. We develop quality in cooperation networks. The starting point is that everyone knows the principles of quality work. We commit ourselves to the continuous improvement of quality by giving emphasis to expertise, monitoring learning results and client satisfaction, and by forecasting future expertise-related needs. High-quality operations make everyday work smooth and flexible. The work life representatives and interest groups participate in quality assurance in the planning of activity and as evaluators. We assure quality via systematic assessment. At Seinäjoki UAS, seamless integration exists between teaching and R&D activities. The implementation of the quality policy is followed up with the faculties steering groups in internal performance negotiations as well as in the annual internal auditings and management reviews performed for the UAS Board. Quality issues are dealt with regularly in the steering groups for the performance areas and in the steering groups respective to the performance units. We follow the implementation of the quality policy by means of student feedback, barometers, EFQM indicators and formal complaints. We systematically document quality policy follow-up. The QAS used at SeAMK comprises the following components: A Quality Manual A strategy and operations guidance system Assessments and audits A quality assessment of the main processes Documentation, information distribution and storage of documents. 11 Ratified by the Board of the Joint Municipal Authority and the Council of the Joint Municipal Authority, May 2007.

19 The SeAMK QAS is described in the university s Quality Manual. The Quality Manual is entirely in electronic format and is located in the Share- Point workspace. Thus, it is available to all SeAMK staff and students. Relevant sections of the Quality Manual are also available to those outside the organisation via the Internet ( SeAMK s operations in observing the criteria of the EFQM Excellence Award are described in the Quality Manual. Deployment of the EFQM model occurred during the years The set of indicators applicable to the performance area of the SeAMK QAS is composed of the performance indicators of the Ministry of Education and the university s own indicators, which describe, for the most part, clients perspectives as well as the viewpoints of those acting within the organisation on the quality of operations. The indicators are set into focus for the main processes of instruction/learning and R&D activities. The six instruction process indicators are as follows: attractiveness, dropout rate, degree completion rate, foreign language instruction, education operating margin, study point and degree productivity. The five R&D process indicators are: scope of service functions, scope of R&D functions, third-party funding of R&D, international funding of R&D, and the R&D operating margin. Operations guidance at SeAMK represents an essential part of the QAS, since through it strategic targets are set and their realisation is monitored. The implementation of strategic goals is monitored through annual strategy indicators in, for instance, management reviews and internal performance negotiations. Quantitative goals oriented towards strategic targets are also implemented therein in addition to action-related quality objectives. In the management review, the real-time aspect of goals, as well as the suitability of the indicators, is also examined. Preparation of the budgets is also connected with operations guidance. In the strategic work performed by SeAMK, the Balanced Scorecard model is applied, which was introduced at SeAMK in The responsibilities of the Seinäjoki QAS have been defined as follows: the Board of the Joint Municipal Authority has responsibility for resourcing and the Board of Seinäjoki University of Applied Sciences decides on the quality system. The university President is responsible for the quality of operations and their development as a whole. The managers, university Vice President (instruction), R&D Director (R&D activities) and Administrative Director (Joint Services, YPA) are responsible for operational quality and related development in their performance areas. In addition, the dean of the school has responsibility for quality in the performance units, but the heads of the degree programmes as well as those persons in charge of R&D and

20 EVALUATION AND DEVELOPMENT - Quality and strategic indicators - Evaluations and audits - Processing of feedback - Other indicators - Influence and results (students, clients, partners, region, society) - Conclusions and development measures GOALS AND LEADERSHIP - Values, mission, vision - Quality policy - Strategic goals - National management - Regional management - European higher education and research policy Documentation and information Staff and students IMPLEMENTATION - Responsibilities per process - Operating in accordance with joint process descriptions - Systematic and appropriate application throughout the organisation OPERATING METHODS AND PROCESSES - Joint operating methods and processes specified in the Quality Manual - Responsibilities defined - Operating guidelines and rules - Good practices Figure SeAMK s Quality Assurance System quality have their own roles. The quality steering group leads the quality work and functions at the operational level. The SeAMK quality steering group was established in Its members are as follows: the university President (chairperson),vice President, R&D Director, Administrative Director, Development Manager, SAMO Student Union representative and quality manager (secretary). The quality steering

21 group has appointed a three-member external specialist group, whose composition is as follows: the Quality Manager of the South Ostrobothnia Health Care District, the Development Manager of the Mikkeli University of Applied Sciences and the Managing Director of Mäkelä-Alu Oy. The external specialists are invited to participate in the quality steering group meetings as required. The quality work group of SeAMK was established in 1996 as part of the quality project launched at the time. The work group has a member (responsible for quality) from each performance unit and managers from the university Office. The university s Vice President acts as chairperson. SeAMK hired an acting quality manager in January The main task of the quality manager is to plan the university s quality system and prepare the international auditing to be implemented in the spring of A part-time person in charge of quality functions is a member of the quality work group at the performance unit. The dean of the faculty defines the job description of the person in charge of quality. Increased participation by students in the quality work has been agreed with the steering groups of the performance units at SeAMK as well as increased participation in the functions of the faculty steering groups. The SAMO student union nominates the student representatives for the university s various work groups. Matters intended for decision making are formulated and operational methods are developed in the work groups. Students therefore also have an important role in taking part in planning and preparation prior to the actual implementation stage. In the QAS, students also have an essential role as providers of feedback in the application stage as well as during studies, the period of graduation and the subsequent years.

22 Results. Objectives, overall structure and internal coherence of the quality assurance system The quality assurance (QA) covers many of SeAMK s activities and the QA procedures form a fairly efficient system. Concerning this auditing target, SeAMK s QA system fulfils very well the requirements of the developing stage and it seems that the system is moving toward the advanced development stage. The Quality Policy and the objectives of the QAS The audit conducted in 2006 revealed that SeAMK lacked a clear quality policy. In the audit report the audit group stated that SeAMK needs to develop a clear policy and strategy for its QAS. A quality policy should set out the scope of the QAS, responsibilities for its implementation and review, and a clear indication of the processes that support the system. Since then a quality policy has been defined and is stated in the Quality Manual. In the policy, SeAMK emphasises future orientation and cooperation between interest groups, students, staff and board members. Quality is seen as everybody s responsibility, demanding systematic assessment and continuous development of operations in order to achieve its goals: better education and improved R&D. SeAMK s QAS has been revised since It consists of four interlinking parts: 1. Goals and leadership, 2. Operating methods and Processes, 3. Implementation, and 4. Evaluation and Development. Documentation and information as well as staff and students are related to all areas. The goal of the QAS comprises the vision, values and strategies of SeAMK and the goals of the Quality Policy. The responsibilities within the QAS are defined at six organisational levels: the Board, the President and Vice Presidents and Administrative Director, the quality steering and working groups, the schools, staff members and students. In 2007, SeAMK appointed a three-member external advisory panel as a support body. Each SeAMK unit has a Unit Manager and a part-time Quality Supervisor. In addition, there is a Quality Manager whose responsibilities cover all of SeAMK. The division of responsibilities has been clarified due to the development of the QAS, a simplified electronic follow-up sys-

23 tem and also due to the strategic organisational changes carried out within the last two years. Fewer units help in decision making and enhance quality, thus eliminating delays. In SeAMK there are a lot of combinations of teams for different tasks, each team comprising many people. For instance there are 19 members in the international working group. Participating in meetings may enhance commitment but it would be worthwhile considering the effectiveness (time and money consumed/results) of meetings attended by a great number of people. Processes have developed greatly in the two years. In order to further develop them, SeAMK is now planning to initiate the procedure for auditing the processes. The first process to be audited will focus on international affairs later this year. The audit team considers this to be worthwhile. The audit team also suggests that a point of interest to develop the QAS could focus on the integration of processes. The audit group agrees that the development process of the QAS has been systematic and has been executed according to the recommendations set by the audit group in The involvement and participation of staff, students and representatives from work life and other interest groups have guaranteed the commitment to this developmental process. The attitude among staff, students and external stakeholders seems to be positive towards QA. The high value given to the QA work is also seen in the division of human resources: for instance the unit Quality Supervisors dedicate hours annually to QA work. Overall structure and internal coherence of the QAS In 2006, the audit group suggested that with regard to the activities, actors and responsibilities, SeAMK should examine its QA structure, the various actors and the related activities and processes. Since January 2008, SeAMK has introduced a new and a more transparent organisational structure comprising an Inspection Board, an administration including the Board and the President of SeAMK, and a division into five schools. Each school now has responsibility for its own research and development compared with the previous set-up in which R&D had its own organisation separate from the respective schools. With this new organisation, the conditions for linking R&D to education are better. Furthermore, the implementation and the following up of systematic quality work will now be smoother. SeAMK s main processes and the links to its vision and mission as well as to its quality work are described in the process chart (see Chapter 3.1, Figure 2).

24 Compared with the previous structure in which each school had its own system, the new QAS is common to all schools, and includes guidelines for the different processes. The follow-up system has been strengthened. Internal audits have been implemented since A strengthened system for internal and external audits and evaluations is in place. The number of quality indicators has been reduced, which indicates that an analysis has been made of those indicators that are effective measurements of quality. Finally, the central components for a continuous enhancement of quality are the strengthened annual cycle and the evaluation of the QAS itself every third year. In 2006, SeAMK had many strategies, including the main strategy and separate strategies for various areas. Also the schools and other units had their own strategies. Clarification and renewal of the vision and main strategy for the agreement period has taken place. Now there is only one main strategy and three sub-strategies: Pedagogic strategy, R&D Strategy, and International operations strategy. Balanced Scorecards are considered to be sub-strategies. Separate performance unit strategies have been eliminated altogether. Considerable simplification has taken place and, during the interviews, the staff members and persons in leadership and managerial positions stated that the changes have made strategic development work more manageable than heretofore. The focus of QA had previously been on current operations because the system did not allow integration with strategically important future objectives. Now the focus is more on the future as future orientation is also mentioned as a principle in the Quality Policy. When there are fewer strategies and they are set for three-year periods the staff members are able to focus their work according to these strategies. Also, R&D projects can be signed for three-year periods. The internal audits, the market surveys and internal performance negotiations are tools that can be and are used for strategic purposes. The strategically important conclusions are extracted from the gathered information mainly by the education and R&D steering groups. Strategically minor important information is analysed by the directors of schools and other units and in respective cases the heads of the degree programmes and the managers of the administration office. They are also responsible for using the information. Some information is discussed with the board of the municipality. In the 2006 audit report, the audit group recommended that the interfaces between the QAS, the management and the steering groups should be clarified. After two years of development work, the re-audit group is presented with a functioning system that provides the necessary information and shows the division of responsibilities. As in all evaluations, the indicators play

25 a very important role in analysing the gathered data. During the visit it was suggested that some of the indicators should be reconsidered. This is a sign of the awareness among people in SeAMK that continuous evaluation of the system and its processes, tools, indicators, responsibilities, timetables, etc., is worthwhile. It is a sign of a good quality culture. In 2006, SeAMK s QAS was based on the EFQM model. An important reason for adopting this model was to provide the possibility to make comparisons both nationally and internationally with other institutions. However, a system for following up, such as RADAR Logic, was not used. In the first audit report, the audit group recommended that the individual QA procedures should be organised into a systematic QAS that includes routines for follow-up, such as RADAR Logic. The QAS is still built within the EFQM framework. Based on the reaudit visit interviews, it seems, however, that the role of the EFQM is not very central to SeAMK s QAS. Furthermore, the audit group did not find any evidence that SeAMK was planning national/international comparisons with other institutions using the EFQM model. It found that SeAMK had developed its own system with its own quality components. It seems that the EFQM model is not necessarily needed as an umbrella for SeAMK s QAS. The re-audit group suggests that SeAMK continues building and further developing its own system around the present chosen components of quality variables, for instance quality indicators, barometers and evaluations. Internal audits as a tool to develop the units In the 2006 audit report, the audit group recommended that an internal peer review should be performed regularly for all the schools and other units at SeAMK. In 2008, as part of the QAS development, SeAMK switched the assessment/evaluation method from peer reviews to internal audits of the performance units. An external expert was used in the planning of the internal audit process, which is described in the Quality Manual. The internal audits have been conducted twice in the performance units and the process appears to work. The internal auditors are trained by external experts. The EFQM criteria based information is carried out in the performance units prior to the audits. The students are well represented in the process. The internal audit outcomes are presented in open sessions, where the results from different performance units are presented in a similar way. This makes it easier to compare units and share good practices.

26 The development of QA and evaluation competencies of staff One of the recommendations in the 2006 audit was that the QA and evaluation competences of SeAMK staff should be continuously developed. According to the audit group this should aim particularly at developing competence for information analysis and identifying the results of the findings. Based on the re-audit material and the re-audit visit, it can be said that the strategy for developing staff QA competence is through their involvement in planning the system and in its implementation. This type of work enhances commitment to and understanding of the meaning of QA work. Owner, stakeholder and partner knowledge is maintained and developed through seminars. External experts are involved in the QA development work and disseminate the know-how to the faculty. The annual personal development discussions between the leaders and managers and the staff are the main tools for detecting the requirements for further improvement of know-how among the staff. In the SeAMK s QAS one of the four Quartals is Evaluation and Development (Figure 3, page 18). In the Quality Manual one cannot, however, find a path or process description concerning the specific issue of continuous development concerning QA and evaluation competences particularly focusing on information analysis and identification of the results gained through the findings. It would be worthwhile determining and developing the desired expertise in this field in the context of the entire institution. Summary There is plenty of evidence of the development concerning the overall structure and internal coherence of SeAMK s QAS, including the creation and formulation of the common Quality Policy and Quality Manual. In addition, common guidelines, reorganisation and unification of processes, internal audits and management reviews make it possible to detect the requirements for the different developmental tasks of the institution. The system also makes it possible to compare the activities between the units. The internal coherence between processes ( Strategic management and leadership, Instruction/learning R&D, Support services ) and units could be developed further. Currently the interlinkage between processes and units remains unclear. In order to detect the reason for inadequate quality in one area or process (such as R&D), it would be worthwhile to develop a loop from one process to another and from one unit to another.

27 The use of external advisors has helped SeAMK in developing the QAS to its present level. The audit group feels that this is a good practice and also recommends its use elsewhere. External, additional analysis/auditing of the manual and documentation and reporting system has given new insight and direction for the internal work done in SeAMK.. Documentation, including the formulation of a quality policy and the definition of procedures, actors and responsibilities The definitions of the procedures, actors and responsibilities are clear, comprehensive and concrete compared to the situation in the previous QA organisation. The documentation is easily available to all. The QA organisation is well designed. The development stage of this auditing target is developing. Definition and documentation of the QA policy, activities, processes and actors In the first audit report, the audit group stressed the need for a systematic institutional quality manual or a comparable document. The new QAS is now described in one manual common to the whole institution. The Quality Policy of SeAMK is also found in the Quality Manual. The manual and all documents of the QAS are published on SeAMK s intranet. Information about the QAS is given to newly employed staff every autumn. The interviews at the site visit clearly confirmed that knowledge about different processes of the quality system was strong among students and staff at SeAMK. It is obvious that the quality culture is strong at SeAMK. The staff and students also appreciated the creation of one common system for the whole organisation of SeAMK. The persons responsible for quality activities send s to staff members and students to ensure that they respond to the required tasks. In this respect, the Quality Manual does not play a strong enough role as a handbook. It does not give precise instructions: when the different activities will take place, who is the person responsible for each activity, how and by whom will the results be analysed, and how and when will the activities be followed up. Presenting the manual in a more transparent and appealing way would be especially useful for an external visitor or new students who does not have access to personal information from the persons responsible for the different activities. Also, the Quality Policy needs to be better highlighted in the manual.

28 The management of follow-up data and QA information The audit group recommended in 2006 that the management of follow-up data and QA information should be further developed through systemising the reporting more uniformly (data and information processing system). In the two years since 2006 SeAMK has created a coherent Internetbased document store called SharePoint. The SharePoint station contains all of the quality data and there are also separate locations for working groups and for performance units. SharePoint is well known across the administration, the performance units and students. The other document locations in SeAMK are its Intranet and Internet pages. They are being continually developed and students have been granted increased access to the intranet. Both system transparency and documentation systems have evolved. Finding information in the documentation is made easier through the use of hyperlinks. However, many students and staff members still expect important information to arrive via . The role of students in the QAS In the 2006 audit report the group recommended also that the students role as partners in the QA of SeAMK needs development. Instead of merely informing, they should also participate more actively in setting the targets of education and its QA, as well as of the support and other services important to them. Students should also be integrated into the analyses of the information produced by the QAS. SeAMK and the student union, SAMO, should set about developing mechanisms to ensure that the students role and feedback are extended and integrated into the QAS more effectively. In the process of development of the QAS, the students have gained a permanent position in the groups throughout the organisation and their role is genuinely effective. Students are being heard and they have the opportunity to present their opinion. In the working groups, students are influencing the actions taken as a result of the quality measures. The student union and SeAMK educate the student representatives to act in the working groups and in the development of the quality system. It is stated in the Quality Policy, that the starting point is that everyone knows the principles of quality work. However, the Quality Manual does not point out clearly the extent of the responsibility of each student s involvement in the actual quality work.

29 The student union, SAMO, has a recognised and permanent role. It has its own working groups, such as Tuntosarvi and Kopo. SAMO conducts a yearly Skopo-survey on students and the results are delivered to all staff and students. The study results have promoted changes. An electronic student feedback form is in use in most of the study programmes. The opjapa electronic course feedback system is described in the Quality Manual. The level of feedback gathered has increased as the implementation of opjapa has expanded. The opjapa results are gathered into a database where they are clearly presented. Results are only available to each teacher s superior. The final responsibility to respond to the opjapa results lies with the head of each degree programme. Hence, the opjapa results process is not clear enough nor is it transparent. The rapid response of teaching staff the feedback to feedback is lacking. The long-term results from opjapa inquiries could be presented more openly. Although quality documentation exists, it is not easily accessible to the students and given its broad extent neither is it read voluntarily. Clear presentation and student-focused outputs would make it easier for students to understand the extent of SeAMK s quality work. Summary One of the core documents for the quality work at SeAMK, the Quality Manual, has now been created and is used across the whole institution. The Quality Policy, which previously was not clearly defined, is now defined in the manual. Furthermore, the involvement of students was marginal and they were not seen as partners in the QAS. Their involvement has now, however, been significantly strengthened. The reporting from the QAS is now organised through a data and information processing system, SharePoint, that all staff and students have access to. Quality work is an ongoing process of improvement; this is also true for the presentation of steering documents such as the Quality Manual. The reaudit group sees that the next important step for SeAMK would be to improve the manual and to convert it into a tool in which the different parts of the quality work and their interlinks are clearly presented. This would further facilitate the involvement of new students in quality work as well as give better information about the system for people outside SeAMK who are not familiar with the system.