Understanding Educational Research

Chris Alexander

Abstract

In this paper I acknowledge that there is great potential for confusion if the key concepts adopted by authors do not match the reader’s implicit definition of those concepts (noted in Mertens 1998, 24). In an effort to address this problem (i.e. the abuse/misuse of academically ‘respectable’ ‘buzz’ words, I will use research terminology with reference to the particular conceptualisation(s) of specific authors. Crotty (1998, 214) holds that invoking the name of one or other icon to characterise one’s approach raises interesting questions, one of which is, why one wants to do that. Even though in this paper I do not assert that any particular approach is better or worse than any other, I do suggest that as a wide range of analytical approaches have been developed (especially in the 20th century) it might be worth considering a ‘Synthetic’ or ‘Maggie’ approach as in the Sim and Loon (2002, 6-7) sense (i.e. picking and choosing or mixing from a catalogue of theories). Richard (2000, 934) also takes a similar stance, she discusses mixing genres (i.e. drawing freely from literary, artistic and scientific genres often breaking the boundaries of each) arguing the scholar therefore might have different takes on the same topic. She (ibid) defines this process as the ‘postmodernist deconstruction of triangulation’. It is held (Richard 2000, 938) that the ‘blurring’ of humanities and social sciences is ‘trendy’ because blurring coheres more truly with the life senses and learning styles of so many. The text I have used in this paper is Gillian Wigglesworth (2001) ‘Influences on performance in task-based oral assessments’.

Overview

1. Unadulterated Positivism. 2

2. Wigglesworth’s research into influences on

performance in task-based oral assessments. 3

2.1 Research Questions. 3

2.2 Method and participants. 3

2.3 Instrumentation and procedure. 3

2.4 Results—major findings. 4

3. A discussion of the main weaknesses of the research. 4

4. Researcher values/attitudes that may have influenced the research. 4-5

5. Suggested alterations to Wigglesworth’s positivist approach. 6

5.1 Justification for eliciting more qualitative data. 6

5.2 Suggested improvements to Wigglesworth’s positivist research. 6

5.3 The emancipatory paradigm. 7

5.4 The constructivist paradigm. 7-8

5.5 Introspective research (my term). 8-9

6. Conclusion. 9

7. References 10-11

Appendix one 11

1. Unadulterated Positivism

Edgar and Sedgwick (2002, 203) hold that objectivity in the natural sciences is an indispensable notion and it presupposes that there is a real external world which is independent of our knowledge of it i.e. the concept is that it is possible to describe this world accurately. Richard (2000, 925) notes that given to science is the belief that its words are objective, precise, unambiguous, non-contextual and non-metaphoric. Wigglesworth seems to believe in objectivity in the above senses. She uses (or maybe hides behind) a ‘façade’ of fairly complex statistical analyses possibly to convince her audience (and maybe also herself) that objectivity exists, i.e. the world is objective and exists independently of knowers. The nature of reality (the ontological question) for Wigglesworth is positivistic in the Mertens (1998, 8-10) sense i.e. one reality exists and that the researcher’s job is to discover that reality; Guba and Lincoln (1994, 109) call this ‘naïve realism’ i.e. it is held that ‘research can, in principle, converge on the ‘true’ state of affairs’. She appears to see research as pure technology or technical process; she could be a ‘believer’ in the ‘divinity’ of research in the Usher (1996, 14) sense i.e. it is held that there is a powerful tendency to think of scientific method as universal and ‘made in heaven’.

Crotty (1998, 26) draws attention to the fact that the meaning of the term ‘positivism’ has changed and grown over time. Mertens (1998, 7) states that positivism is based on the rationalistic empiricist philosophy that originated from Aristotle, Frances Bacon, John Locke, August Comte and Emanuel Kant, though the underlying assumptions include the belief that the social world can be studied in the same way as the natural world i.e. value-free methodology and causal-related explanations. Postpositivists, however, noted in Mertens (1998, 10), recognised certain suppositions required for scientific research were not appropriate for educational and psychological research. Heisenberg (quantum theory), noted in Crotty (1998, 29), asserts that the very act of observing a particle changed that particle. Popper’s Principle of Falsification (1963) noted in Crotty (1998, 31-34) is another postpositivistic paradigm which recommends that scientists should make a ‘guess’ and then try to find themselves unable to prove their guess wrong despite strenuous efforts to do so (noted in Crotty 1998, 31). Feyerabend (1993), cited in Crotty (1998, 37) even asserts that findings are no more than beliefs and we should not privilege them over other kinds of beliefs---even voodoo. Mertens (1998, 8-10) notes that in postpositivism a reality exists but that it can only be known imperfectly because of the research limitation. Even though Wigglesworth’s assertions are tentative, her research design does not appear to be post-positivistic; it does not question the tenants of positivism in a radical fashion as Popper, Feyerabend, or Kuhn did. It does not have a critical-realism ontology in the Guba and Lincoln sense (1994, 110) i.e. reality is assumed to exist but to be only imperfectly apprehendable because of basically flawed human intellectual mechanisms and the fundamentally intractable nature of the phenomena.

Wigglesworth’s epistemology (i.e. what is the nature of knowledge and the relationship between the knower and would-be knower) is, in my opinion, early positivist i.e. the researcher and the subject of the study were assumed to be independent and do not influence each other as in Guba and Lincoln (1994, 110). A postpositivist epistemology would modify such a belief by recognizing that the theories, hypotheses, and background knowledge held by the observer may strongly influence the thing that is being observed (Reichart & Rallis 1994 noted in Mertens 1998, 10). Wigglesworth’s methodology (i.e. how the knower can go about obtaining the desired knowledge and understanding) is experimental from the natural sciences; it, for example, assumes that students can randomly be assigned to conditions (NB this would not be assumed in postpositivist research noted in Mertens 1998, 10). Even though Wigglesworth triangulated her data, the emphasis was not placed on ‘critical multiplism’ i.e. a refurbished version of triangulation as a way of falsifying rather than verifying hypotheses (noted in Guba and Lincoln 1994, 110).

2 Wigglesworth’s research into influences on performance in task-based oral assessments.

The paradigm below summarising the research is partly based on Mertens (1998, 9).

Wigglesworth undertook her research in AMEP centres. The Adult Migrant Program (AMEP) i.e. an Australian education and settlement programme for newly arrived migrants and refugees. Migrants/refugees eligible for teaching receive up to a maximum of 510 hours to enable them to achieve functional English (NB students also learn about the Australian way of life).

2.1 Research Questions

This study was an attempt to report on the kind of impact pre-determined (i.e. by the author) combinations of 5 task variables would have on learner performance in relatively informal classroom based assessment. The variables were: (1) cognitive difficulty of task; (2) whether the interlocutor was a native or non-native speaker; (3) if planning time was made available; (4) task familiarity; (5) structure. The author (2001, 186) justifies her approach by arguing that the majority of work on task variability and its effect on learner language was focused on classroom-based tasks, with few studies examining tasks in the context of assessment.

2.2 Method and participants

80 learners (i.e. AMEP migrants and refugee) from various ESL centres took part in the research project. Five routinely-used, competency-based, AMEP-teacher-developed, evaluative tasks at vocational and proficiency levels were identified. There were four variable-manipulated tasks (i.e. taken by randomly-assigned groups of 20 learners), and a control task, which was thought to be ‘universally-familiar’ to every student, for all 80 learners to take. Trained and experienced teachers administered the tasks. Student feedback on task difficulty was sought by using a five-point Likert scale. The interviewers were thought to be familiar with the rating scales and performances were randomly double-rated.

2.3 Instrumentation and procedure

Three discrete quantitative calculations were made to ascertain the difficulty level of each task: (1) an analysis of variance and T-tests on the rater raw scores measured subject performance; (2) A Rasch analysis for four facets was undertaken using the statistical modelling program FACETS; (3) learner feedback regarding task difficulty was elicited.

2.4 Results—major findings

· Tasks seemed easier when the interlocutor was a NNS. Wigglesworth (2001, 204) believes three reasons might explain how NS-interlocutors might make a task more difficult: (1) learners may be more relaxed with NNS; (2) raters may compensate for a perceived disadvantage in having a NNS interlocutor; (3) learners’ language might be less complex with NNS interlocutors

· Planning time appeared to adversely affect performance in both structured and unstructured tasks possibly because planning time encourages learners to attempt (i.e. on the whole unsuccessfully) to use more complicated language.

3. A discussion of the main weaknesses of the research

Even though this is an ambitious research project with impressive data analyses, reducing the number of variables and considering/explaining more carefully how these variables should be grouped would have made the data analysis more straightforward. The following points may affect the reliability and validity of the findings.

· The data are held (2001, 186) to be pertinent to informal classroom assessment, though the method of assessment seemed formal. It was not clear how this test was perceived by the participants: did they take it as seriously as a formal high-stakes examination?

· Wigglesworth did not explain why other learners were chosen to be NNS interlocutors and on what grounds they were chosen.

· Wigglesworth did not consider how physical characteristics, psychological factors and experiential characteristics played affected these results.

· Please refer to appendix one for more weaknesses regarding the data analysis.

Even though this study could be replicated and may have had some impact on AMEP policy and practice, the positivist research philosophy adopted has implications for the data that was elicited. In section 3, I have drawn attention to some weaknesses in Wigglesworth’s ‘positivist’ methodology; I however do hold a ‘positivist’ contribution to knowledge has been made.

4. Researcher values/attitudes that may have influenced the research

In section four I will discuss two issues that may have affected this research. The first, is, how was this research funded and why? The second point pertains to the fact that Wigglesworth did not consider the stress migrant/refugee students may experience in such language programmes.

Young et al (2002, 215) hold that a new mood characterises the funding and execution of social research i.e. it has taken a ‘utilitarian turn’, a return to an expectation that social science should be useful. Usher (1996, 29-32) warns it is not enough to adopt the unquestioning position that research is simply disinterested pursuit of truth. He further states (ibid) that placing stress on the seeking of a ‘truth out of many truths’ fails to consider the workings of power within the research process. He poses (1996, 32) a reflexive question i.e. what is this research silent about? Race, gender, class relationships?

Wigglesworth (2001) did not mention that she was affiliated with AMEP in her paper; she only briefly mentions (p. 206) in a footnote section that the research was carried out as part of an NCELTR special project funded by the Department of Immigration and Multicultural Affairs. An Internet search, (http://www.nceltr.mq.edu.au/) revealed that NCELTR stands for the Centre for English Language at Macquarie University. On her Internet home-page biography site, (http://www.linguistics.unimelb.edu.au/contact/staff/wigglesworth.html), however, Wigglesworth mentions she had been working at Macquarie University between 1995 and 2001 and also at the Adult Migrant English Program Research Centre (i.e. AMEP) from 2000-2001. Why did the funding for the project go to Macquarie University and not AMEP? Was this disinterested research? Was the ‘truth’ that Wigglesworth ‘chose’ to discover biased by her affiliation with AMEP? Did/does the Department of Immigration and Multicultural Affairs have an agenda with regard to migrants or refugees? On the following Australian public-document page http://unpan1.un.org/intradoc/groups/public/documents/un/unpan000211.html (section E) for example, it is stated that

‘the immigration policy debate is shifting inextricably towards economic considerations, which means that as unemployment is likely to remain unacceptably high and the immigration policy domain will become correspondingly important’.

Did research funders expect the ‘truth’ to be relevant to ‘their’ agenda? How would for example Wigglesworth’s, in my opinion, ‘debateable’ ‘planning time’ finding (i.e. giving students less planning appeared to make tasks easier) fit into this possible ‘agenda”? David (2002, 213-214) states that in the last decade especially in Australia, the notions of ‘evidence’ and social science research have often been elided with political movements for social and political change. He also maintains (ibid) that there have been many contradictory approaches to what counts as evidence and how it may be used to modernise governance and build public policies for the future.

Edgar and Sedgwick (2002, 1) hold that cultural studies have sought to draw attention to issues of race, class and gender. Why Wigglesworth, a white female researcher, did not consider the need to address such issues in her research is interesting; did she feel such data would not be of interest to the funders or was she not interested in this herself? According to Marxist theory the dominant class (noted in Edgar and Sedgwick 2002, 47) in a capitalist society (the bourgeoisie) is strictly in opposition to the systematically exploited proletariat (i.e. those that have to sell their ability to work). I however feel that in this ‘globalised’ ‘multi-nationalised’ world there may actually be two extremes in a continuum, migrant/refugee labourers clearly being at the lower ‘proletariat’ end of the continuum. Wigglesworth in her research did not consider the more qualitative issues appertaining to how threatening AMEP migrant/refugee students might perceive such a test or the programme in general i.e. being an ethnic minority underclass to be mainly employed in low-income service-related occupations with little hope of breaking out of the resulting and unavoidable vicious cycle of poverty that sadly has become synonymous with such work. Why did Wigglesworth not consider the stress such migrant students might be going through while undergoing the AMEP ‘socialisation’ process (i.e. the process whereby the individual learns to be a member of a particular society or culture noted in Edgar and Sedgwick 362)? Could such stress or worry not be considered valid variables in task-based AMEP learning and so worthy of analysis and relevant to the funders? With regard to ethical research issues, Wigglesworth did not mention whether the participants had agreed to participate in the study through voluntary consent forms (i.e. without undue threat or inducement) as Mertens (1998, 24) recommends.

5. Suggested alterations to Wigglesworth’s positivist approach.

In this section I will: (1) draw attention to some problems concerning quantification and present

some persuasive arguments for eliciting qualitative data as well; (2) propose a synthetic approach

based on different inquiry paradigms. The advantages and disadvantages of paradigms will be

considered.

5.1 Justification for eliciting more qualitative data.

Guba and Lincoln (1994, 106-107) note that in recent years strong counter pressures against quantification have emerged. The term (ibid) ‘context striping’ is used to refer to the issue of not considering other variables that may, if allowed to exist, change the research findings. Since such exclusionary designs have implications for the applicability/generalisability of the research, it is argued (ibid) that eliciting more qualitative data can redress the imbalance by providing contextual information and rich insights into human behaviour (e.g. the ‘emic’ view). Padilla and Lindholm (1995) noted in Mertens (1998, 24) maintain that the nomothetic approach seeks confirmation of general laws and uses procedures that parallel the natural sciences, in contrast, the idiographic approach seeks to uncover a particular event in nature or society; idiographic data would be more qualitative and particularistic. Guba and Lincoln (1994, 106-107) hold that quantitative normative methodology, i.e. ‘science’, is privileged over the insights of creative and divergent thinkers and assert (ibid) that qualitative inputs may redress the imbalance; it is also noted (ibid) that some critics of the ‘received view’ suggest using alternative paradigms that involve not only qualitative methodology but fundamental adjustments in the basic assumptions that guide the research. In light of the above, it is suggested that more qualitative data be elicited within the context of an inquiry paradigm shift (i.e. the metaphysics or basics beliefs change) and it is also held that since there appears to be no ‘one truth’ that human beings can readily apprehend, a combination of inquiries (i.e. a synthetic approach NB see the abstract) may provide a variety of ‘takes’ all of which could be relevant to this research project, and maybe even to its funders. I therefore maintain that the positivist research undertaken by Wigglesworth should only be one of the ‘ingredients’ of the ‘research mixture’ or parts in such a ‘synthesis’.

5.2 Suggested improvements to Wigglesworth’s positivist research.

In section three, I drew attention to a number of weaknesses in Wigglesworth’s research design and data analysis. My main recommendations were to reduce the number of research variables and to consider more carefully the data analysis. However considering a post positivist design instead of/or in addition to the positivist, as in the Guba and Lincoln (1994, 110) sense, with a critical ontology might provide an interesting perspective on the reality being observed. The epistemology (modified dualist/objectivist) as in Guba and Lincoln (ibid) would abandon dualism as not possible to maintain; replicated findings would probably be true but would be subject to falsification. The methodology (modified experimental) would emphasise ‘critical multiplism’ as a way of falsifying rather than verifying (as in the Guba and Lincoln sense ibid). However, whether the funders would be interested in knowing that a ‘certain’ hypothesis had been falsified as opposed to being informed about ‘hard positivist-research facts’ is not clear.

5.3 The emancipatory paradigm

In section four, I argued that Wigglesworth did not consider race, class, gender as variables and that there were strong arguments for eliciting qualitative data. Other research paradigms however may be more suited to eliciting qualitative data (e.g. constructivist noted in Guba and Lincoln 1994, 112-113 or the emancipatory noted in Mertens 1998, 8). As cultural relativism holds that there are different standards of morality, practices and belief systems which operate in different cultures and cannot be judged with regard to their worth from a standpoint exterior to them (noted in Edgar and Sedgwick, 2002, 99), an emancipatory paradigm in the Mertens’ sense (ibid), might be relevant to making research more relevant to race, class and gender.

Mertens (1998, 15-19) maintains that the emancipatory paradigm includes critical theorists, participatory action researchers, ethnic minorities among others and is contrasted with Guba and Lincoln’s (1994) critical theory ‘label’. It is held in Mertens (ibid) that the emancipatory paradigm deals directly with the politics in research by confronting social oppression wherever it occurs. One justification for the emancipatory paradigm concerns the realisation that much of sociological and psychological theory was developed from white, able-bodied male perspective and was based on the study of male subjects (noted in Mertens ibid). Was Wigglesworth’s research race sensitive? How many of the participants were males/females? What were their ethnic minority backgrounds? Was there any correlation between gender, race, (class) and achievement in the programme? How did teacher expectations vary depending on the ethnicity, gender, class, sexuality or disability of the students? Was there an asymmetric power relationship between the student and school staff? If so, how did this affect motivation? The above are some introductory emancipatory questions that could become the basis of a different research perspective.

Emancipatory ontology would recognise multiple realities though would critically examine findings via an ideological critique in terms of its role in perpetuating oppressive social structures and policies (noted in Mertens ibid). Therefore, Wigglesworth could also have looked at the way social, political, cultural, economic, ethnic and gender values effected/ affected the programmes. Mertens for example holds (1998, 20-21) that in the case of Oakes and Guiton’s (1995) study, ‘race, ethnicity and social class were used as a basis for signifying a student’s ability and motivation, thus influencing curriculum decisions’. The epistemology of the emancipatory approach would be interactive and empowering to those without power. Mertens (ibid) maintains starting off thought by considering the marginalised; this would expose some of the unexamined researcher assumptions and generate more critical questions (e.g. considering ways research benefits or does not benefit the participants). An emancipatory methodology could combine quantitative and qualitative methods of data collection (noted in Mertens ibid) and provide a valuable alternative perspective on issues relevant to AMEP, however funders may not ‘feel happy’ about funding such potentially long term and ‘controversial’ research.

5.4 The constructivist paradigm

The constructivist paradigm was developed from the philosophy of Edmund Hessel’s phenomenology and Wilhelm Dilthey’s and other German philosophers’ study of interpretive understanding called hermeneutics (noted in Mertens 1998 ,11). Mertens (ibid) states that the basic underlying assumptions of the interpretative/constructivist paradigm are that knowledge is socially constructed by people active in the process of research i.e. the emphasis is that research is affected by the values of the researcher and cannot be independent of them. The ontology (according to Mertens 1998, 11-12) therefore assumes that multiple, and sometimes conflicting, mental constructions can be apprehended i.e. the goal therefore is to understand multiple social constructions of meaning. The epistemology would (noted in Mertens 1998, 13) suppose the inquirer and inquired-into were interlocked in an interactive process, each influencing the other. A constructivist methodology would require qualitative methods of data elicitation such as interviews, observations and document reviews. It is held (Mertens 1998, 14) that reality should be constructed on the basis of the interpretation of data with the help of the participants who provide the data NB certain questions cannot be formulated before the study begins.

Wigglesworth could therefore attempt to elicit data from the programme participants regarding aspects of programme, and then begin to formulate some research questions. She would have to provide data regarding the backgrounds of the participants and the context in which they were being studied. The assumption is that it is the interpretation of the multiple participant perceptions of ‘reality’ that distinguishes this paradigm from the two mentioned in sections 5.2 and 5.3 above. The findings could be of interest to the funders, though this kind of research: (1) takes longer to complete; (2) could be more disruptive to the programme in general; (3) would be more expensive to undertake as it would involve several visits to programme centres around Australia.

5.5 Introspective research (my term)

Richard recommends (2000, 934-938) ‘breaking boundaries’ by considering evocative forms i.e. a blurring of humanities with social sciences can be achieved and should be welcomed by drawing freely from literary, artistic and scientific genre. I believe Wigglesworth could also provide some introspective data about herself in the form of diary entries, essays or even poems. The idea would be to ‘try’ to understand herself better (i.e. not only as a researcher) possibly by writing about the following:

(1) Edgar and Sedgwick (2002, 122) argue that a patriarchal culture privileges a hierarchical way of thinking grounded in a series of oppositions e.g. such as male/female, intelligible/sensitive, active/passive. Sim and Loon (2002, 90-91) use the term binary opposition and hold that discourse in the West in general is based on such opposition; man/woman would be such an opposition. Even though there are many dedicated female researchers in the area of language testing, the degree to which they feel they have to compete with male researchers is not clear. I feel Wigglesworth might consider writing an essay about how it feels to be a (white) female researcher in a patriarchal culture; Richard (2002, 932) notes that graduate students have found the idea of the writing-story (i.e. writing about themselves, their work spaces, disciplines, friends, and families) useful for thinking through and writing about their research experiences. It is held (ibid) that some researchers ‘use the writing-stories as an alternative or supplement to traditional methods chapter and as a link to narratives of those they have researched’.

(2) What drove her to research the things she researched? Wigglesworth could write an account (e.g. an essay or diary account) of what, for example, she ‘really’ felt of the funders and/or whether she thought they had a ‘hidden agenda’. It could also attempt to delve into her innermost concerns as a researcher and/or her desire to acquire public recognition as a researcher. I believe there is not enough of this kind of very personalised qualitative ‘introspective’ feedback accompanying positivistic or post-positivistic research.

These two ‘unusual’ introspective suggestions may provide some extremely interesting supplementation to ‘traditional’ research. Even though following up these suggestions may help Wigglesworth clarify certain issues, presenting such ‘data’ publicly may have unforeseen consequences for the researcher.

6. Conclusion

In this paper I have suggested that mixing an improved version of Wigglesworth’s positivist research with post-positivist, emancipatory, constructivist and introspective (my term) approaches will provide different perspectives and therefore ‘takes’ on ‘our’ intangible reality (i.e. reality appears to be intangible for humans).

With regard to ‘our’ intangible reality, even though life experience and cultural background certainly affect the way we interpret ‘social situations’, the way we interpret ‘things/objects’ may actually, in part, be preconditioned by evolution i.e. our minds—the black box—receive complex data through our ‘imperfect’ and ‘limited’ senses; data is then interpreted almost instantaneously using ‘algorithms’ unknown to us. If our senses were more acute and if the algorithms were ‘better’ maybe ‘reality’ would be quite different, maybe there would only be one reality.

7. References

Crotty,M. (1998). The Foundation of Social Research. London: Sage Publications

David, M. (2002). ‘Themed Section on Evidence-Based Policy as a Concept for Modernising

Governance and Social Science Research’. Social Policy or Society. 1:3, 213-214

Edgar, A and Sedgewick, P. (2002). Cultural Theory, the Key Concepts. London. Routledge

Feyerabend, P. (1993). Against Method. London: Verso

Guba, E. and Lincoln, Y. (1994). Competing paradigms in qualitative research. In N. K. Denzin

& Y. S. Lincoln (Eds.), The handbook of qualitative research (pp. 105-117). Thousand Oaks, CA: Sage

Mertens, D. (1998). Research methods in education and Psychology. London. Sage Publications

Oakes, J., & Guiton, G. (1995). Matchmaking: The dynamics of high school tracking

decisions . American Educational Research Journal, 32(1), 3-33.

Padilla, A. M., & Lindholm, K. S. (1995). Qualitative educational research with ethnic

minorities. In J. A. Banks & C. A. McGee-Banks (Eds.), Handbook of research on multicultural education (pp 97-113) New York: Macmillan

Popper, K. R. (1963). The Logic of Scientific discovery. New York: Basic Books.

Reichardt, C. S. & Rallis, S. F. (1994), Qualitative and quantitative inquiries are not

incompatible: A call for a new partnership. In C. S. Reichardt & S. F. Rallis (eds), The qualitative/quantitative debate (New Directions for Program Evaluation, Vol 61 pp 85-91). San Francisco, CA: Jossey-Bass

Richards, L. (2000). A Method of Inquiry. In N. K. Denzin & Y. S. Lincoln (Eds), The

Handbook of qualitative research (pp 923-948). Thousand Oaks, CA: Sage

Sim, S., & Loon, B, V (2002). Introducing critical theory. Cambridge: Icon Books UK.

Usher, R. (1996). A critique of the neglected epistemological assumptions of education

research. In Scott, D. and Usher, R. Understanding Educational Research. London: Routledge

Wigglesworth (2001). ‘ Influences on performance in task-based oral assessment’. In Bygate, M.,

P. Skehan and M. Swain. (eds) 2001. Researching pedagogic tasks: second language learning, teaching and testing. New York: Pearson Education.

Young, K., Ashby, D., Boaz, A., and Grayson, L. (2002). ‘ Social Science and the Evidence-

based Policy Movement’. Social Policy and Society. 1:3, 215-224

Appendix one

Some additional weaknesses in Wigglesworth’s research

· Wigglesworth did not explain what an ‘unmanipulated’ task was. Furthermore it was stated (p194) that there were no significant differences for learners participating at either level in these baseline tasks (i.e. in the non manipulated tasks). I find this result surprising as it would suggest that the students were all at the same language level (which is unusual at any ESL centre), yet Wigglesworth did not mention that the learners were same-level learners or on what basis they were chosen; she only stated (p191-192) that they ‘were drawn from different ESL centres. An indication of the level of tasks was given on (p206) If they were drawn randomly how was it possible that they were all at the same linguistic level? I suppose, at least for the purposes of the research, it would be important to ‘ensure’ that there were no differences in the control-task data, as ‘differences’ at this stage in the research would raise questions about the usefulness/relevance of the rest the research data. I think presenting all the data for tasks one would have been of interest.

· There were two level types: functional level of proficiency (tasks 1-3) and vocational level of proficiency (tasks 4-5). Yet the use of these terms were not defined or justified in detail and how tasks were level-grouped seemed debatable. For instance (P195) giving instructions about how to use a bank automatic teller machine was seen as functional, yet explaining to a 12-year-old child how to change a light bulb was seen as vocational (and not functional).

· With regard to the five-point Likert scale for student feedback, even though (p206) some categories had been ‘collapsed’ with others, I felt the ‘easy’ and ‘OK.’ categories presented in the data analyses were vague; what is the semantic difference between ‘easy’ and ‘OK’ and would a learner not interpret these words a slightly synonymous?

· With regard to task type one (p 195) it was stated that the more familiar task without planning was the easiest i.e. task 3, yet in my opinion student feedback indicated that task 2 (familiar + planning) was the easiest.

· In task type three the student feedback partly contradicts (i.e. more students find this easy) Wigglesworth’s finding regarding the NNS interlocutor making the task easier