An Evaluation of a Diploma in English Programme : Students ' Performance in Industrial Training

Programme evaluation is an essential part of any specific teaching and learning programme. The results of the evaluation, among others, would inform the relevance of the courses offered to the students in relation to their future in the working world This research study was an evaluation conducted to determine the extent to which the structure of a Diploma in English programme prepares students for the working world as well as the extent to which the graduates produced meet the requirements of the industry. The evaluation was carried out by investigating the performance of 69 students undergoing their industrial training. The data were collected through surveys and interviews. In addition to the students, 35 site supervisors were also surveyed and interviewed. The results revealed that the Diploma in English students were prepared for the working world and they had the qualities required by the industry. The results of the study also highlighted some issues on the additional courses needed for the betterment of the programme. Based on the findings of this research study, some changes were introduced in the programme which enforces the significance of the evaluation conducted.


Introduction
The success of a programme depends largely on the fulfillment of the specified aims and outcomes.In order to determine whether or not the programme aims and learning outcomes are achieved, students' performances need to be measured, not only in the classrooms but also in real life contexts where they have to apply all the skills and knowledge acquired.In designing a course, one of the most important elements to consider is the goals and objectives.Goals and objectives are related to each other in that objectives are normally set to ensure the achievement of the goals.Graves (2000) claims that goals and objectives have a cause and effect relationship.In other words, the goal of a course may be achieved (the effect) if objectives are formulated to support the goal (the cause).At the end of a course, it is important to determine whether the goals and objectives have been achieved and this is when evaluation takes place.According to Kiely and Rea-Dickins (2005) evaluation is part of a process to determine students' learning achievements which are related to a course's goals and objectives.If the students manage to achieve the goals and objectives set for them, it can be suggested that the course is a successful one.In the context of this research study, students' performances during industrial training may indicate the extent to which the specified goals and objectives of the programme have been achieved.This is because during the training, students may need to display the abilities listed under the programme outcomes to carry out the tasks assigned to them.Failure to do so may suggest that the programme fails to prepare students to achieve its outcomes.This is in tandem with the claim stipulated by Metcalfe, Aitken and Gaff (2008) that an education program evaluation may assist to reveal the impact of the programme.In order to evaluate the Diploma in English programme in focus, surveys and interviews with 69 students and 35 site supervisors were conducted.The findings indicated some positive feedback on the quality of the students.In addition, based on the results of this research study, some changes were introduced in the programme which enforces the significance of the evaluation conducted.
This paper will continue with the literature review and the contexts and the participants of this research study.It will be followed by the research methodology and the presentation of the results and discussion before it ends with a conclusion.

Literature review
Evaluation was the basis for the identification of strengths and weaknesses in the curriculum, followed by replanning, implementation and evaluation (Gredler, 1996).Evaluation is a process we carry out to obtain data to determine whether to make changes, to make modifications, eliminations and/or accept something in the curriculum (Ornstein & Hunkins, 1998).Thus, a systematic and perpetual evaluation is a must while determining future strategies.According to Brown (1989) continuous evaluation is important as there should always be preparation for revision of all of the elements in the curriculum plan.He points out the importance of evaluation and states, …the ongoing program evaluation is the glue that connects and holds all of the elements together.Without evaluation, there is no cohesion among the elements and if left in isolation, any of them may become pointless.In short, the heart of the systematic approach to language curriculum design is evaluation-the part of the model that includes, connects and gives meaning to all of the other elements.(p.235) Cronbach (1991) makes a distinction among three types of decisions which requires evaluation: 1) Course improvement: deciding what instructional materials and methods are satisfactory and where change is needed.
2) Decisions about individuals: identifying the needs of the pupil for the sake of planning his instruction, judging pupil merit for purposes of selection and grouping, acquainting the pupil with his own progress and deficiencies.
3) Administrative regulation: judging how good the school system is, how good individual teachers are, etc.Based on the above distinction, the evaluation carried out in this research study is meant for course improvement.Lai, Johl and Abdul Karim (2011) believe that industrial training plays a very important role in an academic programme.This is because it provides students with the opportunities to experience real-life working situations.More importantly, industrial training allows students to apply their academic knowledge to actual workplace environment.In other words, the relevance of the academic courses attended in the university may be revealed by students' performances during industrial training.This in turn will become an indicator of the effectiveness of the programme in producing graduates who possess the skills and knowledge valued by the industry.
Ayarkwa, Adinyira and Osei-Asibey, (2012) conducted a study on 120 organizations in Ghana and these organizations believed that industrial training exposed students to real work environment and helps them to apply theory to practice.The results of this study had also led to some improvements to several university programmes in Ghana.This is because, the evaluations on the students' performances indicated the need to reform the courses offered in the universities to match the needs of the organizations.
A study by Chan and Muhamad Syukrie (2009) revealed that the evaluations on students' performances by site supervisors during industrial training had helped course designers to make improvements, not only on the implementation of the industrial training, but also on the structure of the whole programme.This idea is also supported by Harris et al. (2007) who claimed that students' industrial training results could be used as a basis to improve an academic programme.Evaluations by site supervisors, according to them, may help course designers to decide the knowledge and skills which needed to be emphasized in the classrooms so that students would be able to perform well, not only during industrial training period, but also when they graduated and started working.
According to Peacock (2009), evaluation of these programs is the starting point on the way towards professionalization.It also contributes to program improvement.Therefore, a systematic evaluation should be placed at the very heart of any program (Rea-Dickins & Germaine, 1998).As such, this research intends to investigate the performance of a Diploma in English programme students during their industrial training period as part of a programme evaluation to answer the following research questions; 1. To what extent does the structure of the Diploma in English programme prepare students for the working world? 2. To what extent do the students of the Diploma in English programme fulfill the requirements of the industry?

Context and participants
The Diploma in English programme is one of the programmes offered by the English Language and Literature Department, Faculty of Languages, Sultan Idris Education University, Malaysia.The programme was designed to fulfill the needs for human capital who are proficient in English.Specifically, the programme aims to produce graduates who are able to: 1. acquire the four English language skills (listening, speaking, reading and writing), 2. acquire life-long learning skills, 3. acquire critical and creative thinking skills, 4. acquire ICT skills, and 5. work in various related fields.
Under the programme learning outcomes, it is specified that at the end of the programme, the graduates should be able to: 1. use the English language to communicate for social and professional purposes, 2. apply ICT skills in professional contexts, 3. practise professionalism and ethical values, attitudes and behaviours, 4. fulfill various social obligations, 5. communicate effectively and work as a team, 6. manage information efficiently and practise lifelong learning skills, 7. think creatively and critically, and 8. give responses to various current and global issues.
The results of this study are particularly significant to the Department of English Language and Literature, Faculty of Languages and Communication, Sultan Idris Education University.
Firstly, the results will inform the relevance of the courses offered to the students in relation to their future in the working world.The objectives of the programme suggest the nurturing of various skills such as communication skills, ICT skills and other professional skills, which are all vital in the working world.When the students are undergoing their industrial training, they would need to apply these skills to complete various tasks entrusted to them.Based on the feedbacks from both students and their site supervisors, it can be determined whether the courses taught during their semesters in the university have helped them to develop these various skills.
Secondly, and perhaps more importantly, the results of this study would inform whether the students of the Diploma in English programme have the qualities that make them desirable to the future employers.This is an extremely important information to the programme designers and instructors.The feedback from the site supervisors may reveal whether or not, 1) the trainees are able to carry out the tasks given to them and 2) the trainees possess the traits which are deemed important in the working world.Many programmes in any universities are designed to prepare students for their chosen careers.
However, a good comprehensive academic programme can only be verified when the students actually go out and work.The industrial training is the first step towards actual working experience.As such, information gathered during the training is vital in determining the quality of the academic programme.
The participants of this research study were 69 Diploma in English students undergoing a 14week industrial training course at various government or private agencies of their choices.These 69 students were the total population of the programme's first batch.In addition to these students, this research study also involved 35 officers from the various agencies appointed as the site supervisors for the students.

Methodology
This research is a qualitative study involving 69 Diploma in English students who underwent their industrial training in various agencies and 35 site supervisors.
To answer Research Question 1, data were collected by interviewing students to investigate the extent to which the courses offered at the University prepared them for the working world.
The data for Research Question 2 were collected using the following instruments: 1. Site supervisors' evaluation form of students' performances.
The site supervisor's evaluation form focuses on two main aspects -students' competency level in doing their tasks and students' English language competency level.The supervisors were asked to rate students' competency on both aspects to evaluate the extent to which they fulfill the requirements of the industry.2. Interviews with the site supervisors.
The interviews with the site supervisors were conducted to give the opportunity for them to further discuss their answers in the evaluation form.
The interviews for both the students and their site supervisors were semi-structured interviews (Kvale, 1996).The questions listed were only guiding questions and these questions did not come in this order or in the forms they were written.69 interviews were conducted with the students and 35 interviews were conducted with the site supervisors.Each of these interviews lasted between 30-40 minutes.
The interviews with the students were analysed based on the following codings; a. Tasks they had to do during their industrial training; b.Courses most relevant to their tasks during their industrial training; and c.Courses/skills they wished they had learned before undergoing industrial training.The data from the site supervisors were based on the ratings on the students competency to 1) fulfill the tasks given to them and 2) use the English language to fulfill the tasks.The interviews with the site supervisors were analysed based on the following codings: a. Tasks they gave to the students during industrial training; b.Skills the students had which were important; and c.Skills they wished the students had.

Results and discussion
The results from the interviews with the students will be presented first (Research Question 1) followed by the results from the surveys and the interviews with the site supervisors (Research Question 2).

5.1
Students' interview results (Research question 1) The 69 Diploma in English students undergoing the industrial training were placed in public relations, customer service or training departments of various public and private agencies.The following table summarizes the agencies where the students underwent their industrial training.Based on the table above, it may be suggested that the range of agencies where the students underwent their industrial training was quite varied.This may be an indicator of the trust these agencies had on the students' abilities to fulfill their tasks.
The interviews with the students revealed that during the industrial training, they were involved in doing tasks that include: 1. translating documents (English -Malay or Malay -English), The students were able to carry out the tasks given to them because they could rely on the knowledge gained from many courses they took during their semesters in the university.The courses they found most relevant are: 1. English Grammar, 2. Reading Skills, 3. Listening and Speaking Skills, 4. Business English, 5. English Language Camp, 6. English at the Workplace, 7. All the computer courses offered i.e.English for ICT, 8. Organizing Meetings.
According to the students, other courses such as the literature courses, even though not used directly in doing their tasks, had helped them to be creative and innovative.The courses had also helped them to become proficient English language users which were very important to them.During the interviews, one of the students stated, I think all of the courses I learned are important, and good for me.Some I used directly, like Business English, writing memos, reports.Of course Reading Skills, Listening and Speaking, helped me a lot.Other courses like Literature made me good in English because I have to read and interpret a lot.This student's comment resonates the comments made by other students interviewed.The students were able to play their roles well as they were exposed to relevant knowledge and skills when they were undergoing the Diploma in English course.
In addition, students also commented on how their experience in the classrooms had helped them during their industrial training.The Diploma in English programme, according to them, offered a variety of courses.The nature of those courses required them to do many presentations, to go out and meet people these helped the students build their self-confidence.The experience also gave them the edge to develop social skills which they claimed was extremely important during industrial training.One of the students stipulated, For the English Language Camp course, we had to meet people, ask for sponsorships to get some money for the programme.So, I learned to talk to people, approach people, made me feel confident.Here, when my boss asked me to work during open day, I was not afraid to meet people and explain when they asked about the courses here.
The above comment indicates that the courses offered by the Diploma in English programme not only provided students with theoretical knowledge but it also provided the opportunities for them to develop skills which were important for their future career.
Even though the courses the students took in the university seemed to prepare them for the industrial training, the students expressed the need for a translation course to be offered by the programme.According to the majority of the students, translating English documents to Malay and vice versa was one of their main tasks during the industrial training.Without a proper translation course, they had to rely entirely on their knowledge of both languages which were not adequate.One of the students stated, My main job here is to translate documents, English to Malay especially.I do speak both languages but for translating documents, I think I need a specific course.So I know how to do it more properly.If I had learned translation before, I think I can do a better and may be quicker job here.The student's comment was supported by the claim made by another student who posited, One course I need to make my industrial training a perfect one is translation.Right now I depend entirely on my knowledge of both Malay and English which I think is not enough.I need a proper translation course.These two comments echoed the comments made by the majority of the students interviewed for this research study.Even though they spoke good English and Malay, the students believed in the importance of undergoing specific translation courses.This was to ensure they could produce high quality translation work.This kind of revelation is important for the administrators as it provides the input on how they can improve the programme.
Based on the results of the interviews with the students, it may be concluded that the structure of Diploma in English programme evaluated had managed to prepare students for the working world.

Site supervisors' survey results (Research question 2)
To answer Research Question 2, data from the evaluation form for the site supervisors and the interviews with them were analysed.The form focused on two main issues -the trainees' competency level and their English language skills.Under the trainees' competency level, the site supervisors were asked to rate the students' performance whether it was low, average or high.Altogether there were eighteen items under this section.The emphasis of this section was on students' ability to display some favourable traits which would determine their quality as an employee.These include students' ability to: 1. communicate effectively in English, 2. retrieve and analyse information from different sources, 3. work in a team effectively, 4. make good decisions when performing a task, 5. work independently, 6. display adequate ICT skills, 7. display appropriate interpersonal skills, During the interviews and in the evaluation form for site supervisors, the supervisors were asked to evaluate the students' competency level as well as English language skills.They were satisfied with the students' performance in both aspects.For example, one supervisor rated a student as 'High' for her ability to work in a team effectively.He claimed, She is young, the youngest in the team, but she works well with everyone.Listens attentively, then if she does not know anything, she asked questions.Sometimes she gives ideas, comments, very matured for her age.Very hardworking.We try to make her comfortable and she makes us feel comfortable working with her.She is new but we feel she is with the team forever.We sure are going to miss her when she leaves.That's why I rate her High in the form.Similar comments were also given by the majority of the site supervisors proving that most of the Diploma in English students had the ability to work in a team which according to the site supervisors, was a very desired trait in an employee.
Another trait that was important as an employee was being committed.In the evaluation form, being committed was represented by the items "show concern for quality when performing tasks" and "show the will to succeed when performing tasks" For these items, all of the supervisors rated 'High' for all of the students.These two items reflected the students' commitment in doing the tasks given to them.During the interviews, most of the supervisors commented on how the students showed great interest in their jobs and seemed to strive for a high quality of work.Comments such as " … very committed…", "… very diligent …", "… take his duty very seriously…", "… always consult to ensure job is done properly…" were the most common feedbacks given by the site supervisors when asked to describe their supervisees' work commitment.
Besides being committed, the site supervisors also listed being professional as another quality they desired in their employees.Being professional was defined as having the ability to, among others: 1. communicate effectively in English, 2. make good decisions when performing a task, 3. display appropriate interpersonal skills, and 4. adapt to new situations.All of the site supervisors claimed that the Diploma in English students showed very appropriate interpersonal skills.They communicated well with their colleagues and managed to make themselves became part of the organization.They adhered to the culture of their work place and showed no resistance towards new environment.Even though the industrial training was the first working experience to many of the trainees, it did not take long for them to adapt to the new working environment and this was attributed to their professionalism.One of the site supervisors commented, I was quite surprised when she came.Of course first few days, she was still adjusting, so you can see she's a bit, you know, shy, but after a week or two, she's okay, likes to mix around.She doesn't act -I'm a student, so I don't know.Always willing to learn.When I give instructions, she understand and can do the job.Sometimes, I ask her to do one job only, but she takes the initiative.I add this I add that, very good.
The above comment suggests that the student was professional throughout her industrial training.This professionalism was a good indicator of the Diploma in English programme's success in producing graduates who fulfilled the requirements of the industry.
In addition to being professional, employers also preferred to hire graduates who had good ICT related skills.In the Diploma in English programme being evaluated, the students had to sit for four ICT related papers.They were: 1. Writing for the Media 2. English for ICT 3. Computers and Writing 4. Writing for Multimedia and the Web These four courses were designed to enable students to develop relevant skills in ICT, in addition to the English language skills.Other courses, even though did not teach students ICT related skills directly, they required them to apply ICT skills in finding information or in doing presentations.Such courses included: 1. Business English, 2. English at the Workplace, and 3. Report Writing Having taken all of these courses by the time they underwent their industrial training, the students' ICT related skills were very commendable.All of the site supervisors were very satisfied with the students' ability to apply knowledge of ICT on their everyday tasks.Most of the site supervisors commented that they were impressed with the students' knowledge and skills as they did not have to waste time teaching these students the skills.Even though some of the organizations used very specific computer programmes which were not familiar to the students, they did not take a long time to master the programme.The students' ICT knowledge and skills were great assets which the site supervisors hoped all of their employers possessed.
Besides having adequate ICT related skills, another commendable trait of the Diploma in English trainees was the fact that they were not calculative when it came to the tasks given to them.According to the site supervisors, some employees had the tendency to be calculative of what they did in the office.For example, they only wanted to do jobs which were part of their job description.Anything not in this description was not well accepted.However, most of the trainees showed very positive attitude about this.One of the site supervisors posited, What I like most about her is, she is not calculative.Our office hour is from 9 to 5, but sometimes we have to go back much later in the evening, she never complains.Even when what she has to do is not her job.But she is okay to help.Sometimes I'm afraid that she thinks we bully her so I called her and asked her.She said "I'm okay.I'm learning".So that's very good of her.Employees like this we want.Youngsters nowadays, sometimes very difficult to get job because very choosy, very calculative.But with right attitude, very easy to get jobs.
The above comment gave a very good reflection of the overall attitude of the trainees.This kind of attitude also reflected the quality of the Diploma in English programme as a whole.
In relation to the students English language skills -aural/oral, reading and writing -all of the site supervisors agreed that the students possessed commendable ability.Even though the students did produce errors -be it grammatical or word choice -when they spoke or wrote, the students' command of language were still of higher standard compared to some other trainees they had in their organization.
Based on the analysis of the evaluation form for the site supervisors, all of the students were rated as either 'High' or 'Excellent' for all of the items listed under the English language competency section.This indicated that the supervisors were very satisfied with the students' performance in relation to their English language skills.
Even though there were a lot of good feedback from the site supervisors, they also identified some problems.According to the site supervisors, most of the students were very good at executing tasks given to them but they often failed to provide opinion or generate new ideas especially during meetings and discussions.During meetings for example, the students had the tendency to keep quiet and take down notes without really participating.One the site supervisors commented, She is good, but during meetings she would sit down very quietly, but very good at taking down notes or minutes, very detail but if we ask, do you have anything to say, she always say no.I mean I understand that she's a trainee.So may be that's why.Sometimes they think, I'm a trainee, so better keep quiet, may be that's the reason.
Based on this comment, it may be concluded that students did not really involve in idea generating activities during their industrial training.They were more active and good at performing tasks given to them.There was a possibility that the students were reluctant in voicing out their ideas and opinion because they had self-confident issue.Being a trainee at the tender age of twenty, surrounded by more experienced co-workers made the trainees felt intimidated.Even though this did not affect their overall work performance, the Diploma in English programme administrators may want to look further into this matter.
In addition, some site supervisors also voiced out their concern about the students' ability in performing tasks -especially at the initial stage of the industrial training -as they were not sure what the students had been trained to do.A number of the supervisors thought that the students were trained to become teachers.One supervisor said, Diploma in English students, we are not sure.Are they teachers?If teachers, we cannot take them in but the student explained this and that so I accepted her.So when she finally came, I sat down with her and discuss in detail what she can or cannot do.Only then I know what jobs to give her.So I think, that's one of my concern, in the beginning, of course.This comment suggests that there is a need for the course administrators to come up with some guidelines about the students' nature of training for the potential site supervisors.
Based on the results of the site supervisors' evaluations and interviews, it may be concluded that the students of the Diploma in English programme being evaluated had the criteria to fulfill the requirements of the industry.

Conclusion
This study was conducted to evaluate a Diploma in English in order to discover the extent to which its structure prepared students for the working world and fulfilling the requirements of the industry.The evaluation was carried out by investigating the performance of the students while they were undergoing their industrial training.The results revealed that the students were well equipped to enter the working world and they possessed all the qualities sought after future employers.Positive feedbacks from the site supervisors were true testimonial to the quality of the graduates produced.The results of the study also highlighted some issues on the additional courses needed for the betterment of the programme.Based on the findings of this research study, some changes were introduced in the programme which enforced the significance of the evaluation conducted.

Table 1 :
Agencies for Industrial Training