Manual for doing an educational research

Manual for doing an educational research

Introduction

The idea of doing a research in education is to improve or find new ways of teaching, learning and administration in this sphere. There are different approaches for doing it, but it is possible to identify four main types of required knowledge: description, prediction, improvement and explanation. Good and detailed description is very important for further researchers who will be interesting in using your data; studies made on prediction can be oriented for the future outcome or for avoiding mistakes; improvement approach helps to find a new way of usual methods of teaching or work organization. Explanation type requires more effort in comparison to others. Explanation means the ability to give a good theoretical background of the problem, foresee what can happen later and introduce a way of solution.

It is better always to remember that it is impossible to make such research which will be applicable to all the situations. That is why there will be limitations. Doing a study on education, a researcher will choose participants, for example, teachers' performance in Central Finland and then study background and peculiarities, summarizing in a sample. But even within this region there will be cases which are unique, a kind of exception of the sample. Making a study the research will find reasons and connect them to results which can be in this region or in a whole country, but for other countries this research will not be so much useful as there are other reasons and outcomes. That is why a researcher must be very careful in choosing a title and make limits for himself.

Generally there are two main kinds of conducting a research: qualitative and quantitative. Both of these approaches combine many methods of data collection. Qualitative research is seen as a detailed study of the situation in general, or in a particular city or school, it will be a case study then, while quantitative research in conducted in the same situations but more as a confirmation to the issue. For example a student wants to study teachers' self-evaluation system in Finland. In order to do in qualitative way, he needs to interview talk to people and he will never know to which conclusion he will come in the end; if he decides to choose only one or two teachers and he will need to study their cases more detailed, and it will be a case study then; and for the quantitative research a students will need to use statistical or other numerical data.

Planning a research study

The process of doing a research consists of five parts or stages.

1. Identifying a significant research problem. It is recommended to spend some time for searching for a problem one would like to study. It is very rare that the idea of the research just comes by itself; usually a lot of reading is done before. It can happen that while doing a reading about the problem one wants to study, later he or she will find out that it has already been studied or that it is not very important. The process of identifying can take a lot of time, several months; that is why it is better to start it from the very beginning. While searching for a problem and doing dome readings a research should always keep in mind that the topic of the study must have value, be urgent for now and of great interest for others, otherwise there is no sense of it. And at the same time the topic must be interesting for a researcher as well. A researcher should also think about these kinds of questions: What is already known about the topic? What are advantages and disadvantages of previous studies? What is still unknown? And what will be my impact on it? At this stage there can be an option either to make a theory-based research or the one which will extend previous studies. The theory based research is usually conducted as a confirmation of a hypothesis and gives explanation to already known fact. While the idea of doing a study of extending previous research is to analyze, summarize or go into details, the validity of which can differs according to the time.

2. Writing a research proposal. It is recommended to write it before a student starts the research because it will help to plan it and make limits of the research. Writing a research proposal helps students to formulate their thoughts on the paper, can be used as a guide and needs to be approved by an advisor before collecting a data. Research proposal is a very important document and consists of the following parts: introduction, literature review, data design, methods and analysis, human subject protection and the time line. Introduction is the first or an opening part of a study and is multifunctional: it explains the problem and some reasons for this, what is the connection between the problem and the nowadays literature, what are the hypothesis, research questions and design. There are two possible types of hypothesis. First is a directional one and here a research will show the connection between two facts (for example, the usage of painting in chemistry and physics improves students' knowledge), the second one is null hypothesis, when a researcher states that there is no connection between two factors (for example, the idea of using painting at chemistry and physics classes does not improve students' outcome). But if a research is not testing the prediction (that is hypothesis), he can have some research questions or purposes. It is better to explain to a reader why and how a researcher decided to be interested exactly in these moments. There is no strict difference between questions or purposes; it is just a matter of personal preference. Literature review section is a big one. This part requires a lot of readings, some of which the research has probably done while identifying a problem. In the literature review section one should include findings related to the topic, summarize reading of articles and other publications in order to show how studied the topic is and what are the extremes or different opinions about it. Literature review is an important part of the thesis or dissertation and usually after it almost half of the study is done. Research design and methods are also important parts. In the design section a student needs to say what kind of research it is going to be, explain this decision and how he or she is going to make it within limits of this particular research study. The methods section must be more detailed. Here the research has to choose the ways for data collecting and explain why he or she prefers these methods than other and also prove that these methods are needed and how they will be implemented. Data analysis part must be well thought-out in the beginning of the research. It will give a research feel of time and the problem of lack of time will be avoided. In the data analysis part the researcher has to show the connection between hypothesis and research questions with the chosen research methods and results, and it is recommended to summarize them in different parts so that a reader can easily see them. It is ethical always to keep in mind that the idea of any research is for the human’s well-being and sometimes the advisor or grading committee can ask for some papers proving that nobody was hurt or participants' rights were taken into consideration. Time line is a kind of monthly or weekly plan of making research. This plan will make a student to think about possible difficulties in advance and do some preparations beforehand. As soon as a student has this line, than it will need just to be followed. The students should also think about participants when and how it is better to contact them for collecting data in few months.

3. Conducting a pilot study. This process is related to the data collection part, when it is better to check data collection instrument before doing it in reality. Pilot study is a way to test your data collection methods on few outside participants. This procedure will help to see disadvantages of the plan and make some corrections if needed.

4. Conducting the main study. This is practical part of the research: data collection and analyzing.

5. Preparing a report. This stage does not take place in all the situations, but if it does, it means to give a presentation on the conference or write an article to the journal about the topic. In any case it will be a good point for employee’s CV or resume.

Statistics in the research

Statistics are numerical data got through mathematic calculations which can be used for different purposes. There is common misunderstanding that statistics can be used only in quantitative research studies. But there several ways how these data can be implemented in qualitative studies as well. For example, Collins used some calculations to measure reading abilities of first graders some of them considered to be in high-ability reading group, while other in a low one. Calculation was used to count time duration, percentage, amount of sentences and etc. But the whole study in general was made in a qualitative way and the mathematic measurement was one of the methods.

It is worth remembering that there are several types of scores in the statistics.

Continuous scores

Continuous scores can be found in intelligence or IQ tests. In reality these tests are not never-ending, and have limits. But theoretically there must be a possibility for the results to be continuous.

Raw score is the individual’s result in a multiple item. Usually raw scores do not give enough information. If the students result is 85 out of 110, it is difficult to say if it is good or bad. If it is the highest score in the class, it will be good; or it can be average; or even the worst. That is why derived scores are used more often. In this measurement way the data is interpreted in a comparison to other results.

Age and grade equivalents show already measured students' abilities at certain levels. These data show the average age and level of knowledge students can have in different grades. Usually this equivalent is applicable for tests results. For example a researcher wants to measure students' reading abilities. It is already known that in general people read about 170−200 words per minute, but this average level is not applicable for school students. That is why a researcher must take into consideration participants' age and grade information in order to make objective conclusions.

Rank scores show the position one student or a person takes in comparison to others who have the same experience. Ranking list can be used at schools where all the school-leavers will be ranked according to their grades from the highest to the lowest. The perception of ranking list can vary according to the scale. For example if there are 50 applicants for a position and they are ranked on the scale from 0 till 5, there will be no difference between applicants sharing the same rank, but if the same applicants are ranked from 0 till 50, then much more information will be takes into consideration and the difference within 5 or 10 points can be crucial.

Measures of central tendency in descriptive statistics are used to calculate the average among all the scores. Within these measures it is possible to identify further calculations. Mean is measured by dividing the sum of all possible scores into the score group of students got. Median is what can be found in the middle of score distribution, while mode is the most common amount of scores students get. If the group of students presents on the average level without a big difference of knowledge level, then the median and the mode can be equal. But if there are mane extreme points, then these data will be different. Standard deviation is the most used calculation in statistic data. There is a special formula to calculate it, but it is based on the mean score.

Sometimes after the data collection process a researcher can find out that some data are missing. Missing data means that a researcher planned to get special amount of data from different sources (participants, documents or other information) but either because of carelessness or some unpredictable situations the plan did not work. If the expected participants are students, it can happen that some of them do not come to school because of sickness, there is always a human unpredictadness. In this case if many participants are missing, probably it will be better to postpone the data collection process for few days. It is always better if everything works according to the plan, but if some data are missing, then the idea of the whole research and later results can be useless. But even if some data are missing, the researcher must be honest and write it in the study, but never try to cheat by giving wrong information.

Data analysis process can be always conducted on two levels: multilevel or one level grouping. For example a researcher collected data about students' perception of teacher’s work with seven-graders. On the one level grouping a researcher will concentrate only on these particular students and on one or several chosen teachers. But if the data are analyzed on a multilevel then the researcher has to look at the situation in a wider way and look at teacher’s work on the whole school level.

Processing statistical data is also a long and quite complicated process. Usually in quantitative research studies there is a huge amount of information which needs to be analyzed. More and more often it is done with the help of the computers. Nowadays there are several computer programmes made for data analysis, such as SPSS (Statistical Package for the Social Science) or SAS (Statistical Analysis System) and others. The usage of these programmes is not very easy and it is better to have some experience in working with them. If a researcher is inexperienced, it can take a long time of him to get used to it; that is why some students prefer to search for help by hiring computer consultant. Data analysis process is one of the final stages and it is recommended to check the data for accuracy before submitting it. The data must be checked very carefully, from the very beginning of typing data, and then different formulas for calculation and results.

After the data analysis is finished a researcher should take care of the study and keep it a safe place. It is recommended to have several copies of it and also still save the raw material just in case if later he or she decides to wants to have a look at it or show it for the committee. But it is not common to re-analyze the data, it is better not to make any correction after the process is completed.

Selecting a sample

The selection of participants for the research must be well thought-out. The researcher must be very attentive and be able to give good reasons proving his or her decision. The number of participant can be different in accordance to the research scope. If a researcher wants to study the influence of the national curriculum on schools' performance, then probably several schools in a municipality or even several municipalities will be involved in the data collection process. But if a researcher is interested in some particular cases, such as life of some celebrities, then even one participant will be enough. One of the criteria for judging research results is population validity; it means that results concluded from the research study can be applicable for the population of the city, region or even state. In order to achieve the real validity, the number if the participants must be sufficient. It is worth remembering that these participants must be chosen randomly, without any privilege. A different way of sampling is a purposeful one. The topic if this study has to sound very narrow, such as «Challenges graduate teachers face during their first year». The idea of this research is to know some peculiarities, go into details, and as a result of this the participants will be chosen very carefully, only those who suit the requirements. One more way of arranging a sample is replication logic. It means that a researcher will base the study «to check» or «to prove» one of the previous studies. An example of this can be a teacher’s leadership style. Suppose that according to one of the studies a biology teacher was successful by using a particular leadership style. The next researcher can use this study as a foundation for his or her study. And finally there will be several studies proving that biology teachers in a particular situations or cases can be successful by using a «special leadership style». The biggest quantitative research which can be done is the one where the whole population will be covered. But practically it is hardly possible. That is why there are two common ways: to identify a target population or accessible population. In the target special group of people can be involved, such as all biology teachers in New Jersey, or special teachers in Tampere. The idea of this target group that it sounds not very huge, but the outcome can be applicable for the same group of people but wider. Another way is accessible population, the research done in this way can cover larger group of people, such as all school-leavers in Kuopio. Of course practically it is not possible to involve all the school-leavers, but a researcher has to choose those schools which are close the place where he or she lives and some neigbouring. Then some generalization must be done. First the results from the participants who were involved will be generalized to the accessible population and then it can be generalized to the target group, this process is called inferential leaps. And these generalizations can be done much easier if participants were chosen randomly, if everybody was equal and could participate. But if there was a selection of people for the research, then the researcher has to prove that any generalization is possible, to show that some similarities between participants and accessible population and then target population can be found. Choosing participants for the sample randomly is more challenging than a systematic way. Random selection takes more time, it can be done either through a table, where each possible participant has it is own number or with a help of computer programme. While in the systematic way more logic can be found, such as every tenth or every 37^th student or a teacher.

The idea of choosing a sample in qualitative research is different from a quantitative one, the most common sampling is a purposeful one, within which Patton identifies 15 strategies.

1. Extreme or deviant case sampling is concentrated on unusual or «strange» cases. These example can give some understanding of the typical cases. But the main problem of these studies is that researchers can dismiss some findings.

2. Intensity sampling happens when a researcher chooses the best representatives of the field. They can be the principals of the most successful school or the most famous teachers. Some people can be skeptical about it, because it does not look like their case, but in general this sample will give deepen knowledge and understanding of the situation and maybe even inspire some followers.

3. Typical case sampling involves usual people without any extremes. Usually a researcher will use this sample when a new theory or approach needs to be tested.

4. Maximum variation sampling may work if a researcher wants to see how new curriculum or reforms are implemented. For this purpose he or she will need to collect data from all the possible cases, the more different they are the better. This sample can serve two purposes: to observe how the same proramme can be introduced on different levels and whether the idea of new reform can be cut in different situations.

5. Stratified purposeful sampling usually includes cases on three levels: average, above average and below average. This kind of sampling will give more insight understanding of the phenomena.

6. Homogeneous sampling totally different from the maximum variation sampling. For the homogeneous sampling a researcher has to choose very similar cases in order to give a more detailed and deeper understanding of the case.

7. Critical case sampling concentrates on one or two examples which are different from a common vision of the situation. However this kind of sampling happens mostly in sciences rather than in in education studies.

8. Snowball and chain sampling happens when a researcher asks well-situated people to assist him or her cases to study. Finally a researcher may find many interesting cases to study but choose only some of them.

9. Criterion sampling can involve only those participants who suit the research criteria.

10. Theory-based or operational construct sampling can be done in order to prove some well-known or new theories of the real-world. For example a researcher wants to study whether «Style leadership theory is true or not» According to this theory a person goes through four stages during his or her work-experience and a manager has use different working methods at all these stages. In order to check it a researcher will need to select people from all these four stages and a manager.

11. Confirming and disconfirming case sampling is usually done to check the validity of the previous study. It can be done in two ways: either to select cases which will confirm the previous study or find good candidates for opposite outcomes, i.e. disconfirm it.

12. Purposeful random sampling. The idea of random sampling in qualitative research is different from the same in quantitative. The idea in qualitative study is not to generalize to the whole population, but again to show that the theory’s idea is not wrong or biased.

13. Sampling politically important cases can be used for the researcher’s purposes or funding agencies.

14. Convenience sampling is usually used because of the availability of the cases. However this sample in better to be avoided, because it not very purposeful in comparison to the above discussed.

15. Opportunistic sample is described by Patton as a different one. The idea of this sample is that the selection of participants is based on the previous studied. This way is very important in qualitative studies, because it helps to avoid collecting useless data. This sample is more flexible in comparison to the earlier described, because once a researcher decided whom he or she will involve in the extreme, purposeful random, critical or other sample, it is not recommended to change them, that is why opportunistic sample gives a researcher more chances to find interesting cases.

But sometimes in qualitative or quantitative study a researcher can face a problem that not so many people wish to be involved in the project. And they can give many reasons for this, such as lack of time, do not want to do something according to instructions, do not like the idea of the research or some other reasons. In this case a researcher can come to a conclusion that volunteers are needed. But it is always worth remembering that sometimes if volunteers are not old enough or because of other reasons, the researcher will need to get permission from parents or caretakers. And even if the volunteers are old enough they have to be informed about the study. Usually it can be done by putting a notice describing the idea of the research and also place and time of data collection process.

research study explanation

Tests in the educational research

Test can be one of the ways for data collection. In order to choose a good test, a researcher should remember that it hat to respond six main criteria:

1. Objectivity.

2. Standard conditions for administrating and scoring.

3. Standards for interpretation.

4. Fairness.

5. Validity.

6. Reliability.

In general it is possible to identify two kinds of tests: standardized and local constructed. Standardized tests are developed by the Ministry of education or other state institutions. These tests are usually clear and well written, provided by tables of norms and have a good scoring system. But as disadvantage people point out that it is impossible to have a test which will suit all students' mental ability and will show the student’s real level of knowledge. The problem is that some students even if they are very smart, can experience lack of time, go deep into details while thinking of an answer and others. An alternative to it is a local constructed test. There tests are usually designed by teachers who want to measure students' outcome of the topic or the course in general. The results of these tests can be much higher as a teacher knows the students, their learning style and can make a suitable test for them. But unfortunately these teachers are not aware enough of tests' requirements as they are not taught it in the university. That is why it is not recommended to base a research study on the teacher’s designed tests.

If the data is collected through tests there are three possible ways of the measurement.

1. Norm-referenced measurement or otherwise it is percent measurement. In this case a researcher or a teacher has to calculate the number of correct answers and transfer it into percent. This kind of measurement is usually used on the national levels when general students' outcome needs to be checked. The disadvantage of this method is that it does not give information on students as individuals.

2. Criterion-referenced measurement is based on the student’s result in comparison to the test criteria. This method can be used to grade the course as pass or fail.

3. Individual-referenced measurement can show the student’s learning progress for the period of time. Here a teacher or researcher can see the difference between previous tests and a new one and make necessary conclusion.

Nowadays more and more tests are computerized. This fact made some changes in usual testing system. If it is a computer based test, the students usually can not look through the test and answer the question they know and then come back and think of other question, in some tests students also can not return to the previous page and change the answer; and on the other hand it saves time on checking the results, sometimes they are known immediately and reduces the scoring errors.

It is possible to identify in the educational research ten types of tests which can be combined into two groups: performance and personality measures.

Performance tests:

1. Intelligence tests are usually made to measure person’s intellectual level. These kind of tests will include reading comprehension, vocabulary choice, solving of a mathematic tasks and others. The most common and word-know test is intelligence quotient or IQ test.

2. Aptitude tests are designed to «predict» person’s ability and skills in certain subjects or positions.

3. Achievement tests are aimed at measurement of student’s progress or outcome in certain subjects or areas.

4. Diagnostic tests are a form of achievement test, but concentrated mostly on student’s strengths and weaknesses.

Personality measures:

1. Personality inventories are aimed at self-evaluation of different personality traits. These tests require truth and diligence of the respondents otherwise the results can be wrong.

2. Projective techniques reflect person’s inner thoughts, assumptions and priorities by providing amorphous freedom.

3. Measure of specific personality characteristics tests are designed if an individual wants to measure his or her skills or abilities in specific area or in general.

4. Measures of self-concept tests usually combined with self-esteem measurement, when an individual measures the feelings and thoughts of himself.

5. Measures of learning style and habits tests.

6. Measure of vocational interest’s tests usually includes questions concerning individual’s favorite books, movies, sport activities, hobbies and any kinds of interest.

Method of observation

Another method of data collection is observation. In some cases the most reliable method can be observation. A researcher can not always rely on people’s opinions, answers and self-evaluation, and in order to understand the real situation, it is better to come and see it with your own eyes. But before doing it, a researcher has to decide what his observational variables are. There are three kinds of them: descriptive, when a researcher describes every detail without any interpretation; inferential, can be used if a researcher wants to observe some process, such as explanation of new material, i.e. how a teacher behaves, voice, words and etc; and the last one is evaluative, the same approach as in inferential observation but then some evaluation must be done.

In order to make a good observation, several points of interested can be chosen, but physically a researcher can not concentrate on all of them, that is why four main stages of recording are identified: duration, the main question for this part is how long…; frequency-count, here attention needs to be paid or how often…; interval, the aim of this stage is individual’s behavior during certain intervals of time; continuous stage requires the writing of so-called protocol of everything what is going on the certain moment.

Before the process of observation starts it is necessary to know what kind of recording procedure will be used. It is possible to use some standard forms which were designed by some researchers before and it will save time. But if none of the standard form suits or because of other reasons, a researcher can develop his own form. It is also possible to train some people who will do observation as well. This practice has two advantages: first it will save time on data collection and second if several people are involved in the observation, the data then will be more objective and reliable. In order to have some assistances, before the procedure starts, they need to be trained, explained how to work with observation form and better practice the procedure in a similar situation.

There are some typical mistakes, called observer effects, made during the observation which are better to be avoided.

1. Effect of the observer to the observed. It is natural that people behave differently if there is a person who observes them. In order to avoid observing wrong behavior, it is recommended to come and visit the place and people or students several time, before the procedure begins. During that time participants will get used to a researcher and in few days will act normally. Sometimes if a researcher wants to evaluate the work of a particular teacher, it is better not to say the real aim of the study, because it can influence teacher’s methods.

2. Rating error, can happen if a researcher can not identify what is of great importance for the study and what is which actions are not relevant at all.

3. Observer contamination happens if a researcher has some specific knowledge before which can influence the whole procedure. For example a researcher wants to evaluate the work of English language teachers in several schools. But if he or she knows before that this teacher is better and the other one is one of the worst, then the quality of observation will be different based on this unreliable information.

4. Observer drift and reliability decay can happen mostly to the researcher’s assistances of observation. If the procedure lasts longer than a week or so, they can forget their main target and concentrate on different issues. In order to avoid it, it is better to organize some refreshing meeting with these people.

The process of observation in qualitative research differs from the same in quantitative. First, in the qualitative study it is allowed for a researcher to express his own feelings, emotions and knowledge on the topic, to be less objective. Second, the target of observation in qualitative research is more flexible and can be switched if a new more interesting issue is found out. Third, the observation itself is wider and covers more details. The observer in qualitative study can be either complete observer only or even a complete participant.

Sometimes if the there is no other way, the observation can be made when participants are unaware of the study. In some cases a researcher can ask a teacher for permission to observe students or maybe students can be informed that one day during this week or month some observation will be done, but without identifying the date.

Nonexperimental research studies

Some of the educational studies are called nonexperimental, and can be done either through descriptive or casual-comparative designs. Descriptive research is very important in educational sciences and can be adequately used both in qualitative and quantitative studies. The main question of descriptive research is «What is???» and then a researcher tries to give good explanation of the issue. Making a descriptive study, the researcher has to follow the usual procedure: identify the problem, research hypothesis and questions, choose sample, make data collection and then analyze them.

A researcher should always remember that sometimes there are two kinds of variable: independent and dependent. Suppose somebody hypotheses that introduction of flexible curriculum will increase students' learning outcome. In this case curriculum is independent variable, while students' outcome depends on it.

Another nonexperimental research method is causal-comparative research. For this method a researcher has to select people in whom the independent variable is present or absent and then observe if there will be some changes connected to the dependent variable, in other words, the researcher studies cause-and-effect correlation.

Correlational research is depended on two variables and it usually goes together with correlational statistics. The results of the study are presented in scattergram, where the connection between two variables is shown, xand y-axis correspondingly. There can be either positive or negative correlation. If it is possible to «predict» how the y variable changes according to x, then it is positive; if it is impossible, then the correlation is negative. The use of correlational method allows a researcher to see how different variables can influence each other, what is not so easy to see in other methods, where main conclusions will be about effective and ineffective teaching styles or whatever.

Conclusion and discussion

As it is seen from above, there are plenty of ways to make educational research. Sometimes people are not aware of all the advantages and disadvantages of different methods and how they can be implemented, and mistakenly believe that there are two different kinds of research: qualitative and quantitative and as a result of this the methods are divided correspondently.

But the authors of this book showed how and what for each method can be used in different ways. This book gives a light and has to be a manual for everybody who wants to make research in education.

Nowadays alongside with globalization more and more educational studies are done, international tests are becoming more available, demanded and popular and it requires a good system for data analysis. Majority of these tests are either totally computerized or at least data are analyzed through special programs. And although the results are made through statistics or other «quantitative» methods, the researchers are able to give good interpretation and characteristics. And it proves that education is not a strange science.