ࡱ > n q m d bjbjVV 4 < < \ + j j 8 t m $ [( ' ( ( ( ( ( ( * , D ( ( ( ' ' % h N& bK " z% ' +( 0 [( % , , N& , N& ( ( [( , j s : The challenge to be ethical in evaluation
Dr John Donnelly HYPERLINK "mailto:jsdonnelly@bigpond.com" jsdonnelly@bigpond.com +61 407 405 266
Dr Alana Hulme Chambers
Paper presented at the Australasian Evaluation Society International Conference, Sydney, Australia, 29 August 02 September 2011
Abstract
This paper highlights the often unforeseen hurdles and oversights in the conduct of ethical evaluation. While most evaluation practitioners would consider that they are always ethical in their work and, indeed, the Australasian Evaluation Society (AES) promotes ethical conduct through its published guidelines, do all evaluations adhere to all of the ethical issues that can occur? Given time, budget and other constraints, power and authority gradients between the evaluator/data collector and the informant, methods used in data collection in relation to cultural differences, skills level differences, terms and conditions, knowledge ownership and evaluation context is it possible to always ensure an ethically pure evaluation?
Ethical guidance is provided in the ؿ50Ȼguidelines relation to informants differences and inequalities, informed consent and confidentiality. In the gathering of data for evaluations and the subsequent use and distribution of that data, it is assumed that these rights have been addressed. But what of rights not addressed in guidelines, such as informants rights regarding from whom is the data collected and how is this done, and the rights of the informants in accessing and reviewing the evaluation findings?
With these considerations in mind this paper is focused on the rights of informants and the role of evaluators in relation to the data generated by an evaluation; from whom is the data collected and how is this done, and the rights of the informants in accessing and reviewing the evaluation findings.
A number of scenarios are presented in this paper regarding ethics and informants rights relating to data provided for an evaluation. Options are discussed as to how these ethical issues might be satisfactorily addressed in lieu of specific guidelines.
Introduction
This paper focuses on the challenges that the notion of being ethical presents when working with informants during the course of an evaluation. Whilst ethics in evaluation can include the conduct of the evaluator in relation to evaluation, and/or the moral-political orientation of the practice of evaluation (Schwandt, 2007a:118), this paper is focused on ethics in relation to the awareness of, and subsequent respect and protection of, evaluation informants rights. The aim of the paper is to explore the rights of informants and the role of evaluators in relation to the data generated by an evaluation; from whom is the data collected and how is this done, and the rights of the informants in accessing and reviewing the evaluation findings.
How do we negotiate and address these challenges whilst remaining ethically responsible to the informants who provide the vital information upon which the evaluation relies? Vulnerable population groups in two contexts, international development and community health in a developed country, are referred to in this paper to explore how these ethical issues might be satisfactorily addressed in lieu of specific guidelines.
Setting the context
There is no doubt that ethical considerations are recognised by evaluators as needing to be adhered to when undertaking evaluations. As such, guidelines and principles are clearly articulated in the evaluation profession, as they are for other professions (Schweigert, 2007). Many professional evaluators in Australasia (both those working privately and those employed by organisations) will be not only be familiar with but are likely to regularly utilise the Australasian Evaluation Societys Guidelines for the Ethical Conduct of Evaluations. Contained in these are 22 guidelines covering the commissioning and preparation for an evaluation, the conduct of the evaluation and the reporting of the evaluation. Of these guidelines, only three have reference to the evaluation informants. These three guidelines cover the issues of race, gender, sexuality, physical and intellectual ability, religion, ethnicity and socio-economic background; awareness of the evaluation by the participant; informed consent; and acknowledgement of contribution. The other 18 guidelines are related to issues concerning the commissioner of the evaluation and the evaluator.
Whilst these guidelines present a framework within which evaluators may find guidance relating to some of the common concerns regarding the ethical conduct of evaluation, the guidelines are generally broad and can leave room for varying interpretations of the notion of being ethical (Morris & Jacobs, 2000). This is not unique to the ؿ50Ȼguidelines, with the American Evaluation Association Guiding Principles also been noted as having a similar, abstract content, with little guidance on interpretation and application (Schwandt, 2007b). It could be argued that interpretation is needed in order to ensure that ethical responsibilities are being met whilst, at the same time, evaluations are feasible, useful and appropriate (Schweigert, 2007). However, what is an evaluator to do when the guidelines dont provide suitable reference to ethical issues that have arisen?
The contexts and scenarios presented in this paper do not provide universal solutions in relation to the rights of informants and the role of evaluators in relation to data generated by an evaluation. However, they do aim to alert evaluators to situations where ethical behaviour may appear to be practiced in relation to the gathering, analysis and use of data while in reality it is questionable at best and unethical at worst in relation to some or all informants.
The first context is that of community level evaluation of international development projects. In this context informants are very often illiterate, culture and language can be significant barriers between evaluator and informant and cultural mores within the culture of the informants might mean that certain groups might be excluded from participation and/or representation in the evaluation.
The second context is that of marginalised population groups within the Australian community. Marginalisation can be understood as being on the edge or outer in relation to employment, education, family and social networks, opportunities for health and wellbeing, or political participation (Munk, 2002). Informants that belong to marginalised population groups have varying levels of economic, social, emotional and educational vulnerability; all of which can be unintentionally exploited by a nave evaluator. In order to explore vulnerable population groups within the Australian community, homeless young people are the marginalised population group referred to for the purposes of this paper.
Whilst the different contexts of developing and developed countries must be considered, the informants in both contexts share similarities in relation to vulnerability and imbalances in relation to power and control. As such, the following section explores how ethical concerns regarding informants rights can be addressed for informants in both contexts. Where the cultural and country context is significantly different, options relating to ethical consideration will be discussed separately.
Ethical evaluation concerns in the international development and developed country context
This section discusses two scenarios which can raise ethical challenges for evaluators in both the international development and developed country context. Each scenario is described and then possible options for addressing the scenario, based on the authors experiences, are discussed. The two scenarios are based on the following ethical considerations:
from whom is the data collected and how is this done; and
the rights of the informants in accessing and reviewing the evaluation findings.
Scenario one: from whom is the data collected and how is this done?
Anja has been commissioned by a large NGO in Melbourne to evaluate a health promotion program they have been running for homeless young people. Anja enjoys using surveys; they are easy to administer and can capture easily quantifiable information. She decides to use the surveys in conjunction with individual interviews with homeless young people and decides to conduct these at the NGOs head office in the Central Business District of Melbourne as they have kindly offered her an interview room to use for this purpose. After five no shows and three interviews where the young people have failed to complete the paper survey and said very little of value in the interviews, Anja is perplexed as to what to do next. She decides she will be very firm with the next young person (if they show) and let them know it is their obligation to participate.
Informants have an ethical right to refuse to participate in an evaluation. But what if they dont have the power, confidence or capacity to refuse when asked? Informants also have an ethical right to have input into how data is collected from them. The information they possess often relates to their own lived experience and, as such, has the potential to be intensely personal and important. At the same time, such data is extremely valuable to the evaluator trying to create an accurate and meaningful evaluation report. So what are the ethical considerations that may need to be taken into account when considering how to collect data? Key to this step is determining from whom data is collected.
In international development projects, the informants are nearly always those who have been the target of an intervention. At first glance it may seem obvious that this is the population group from whom data is collected. But is it as simple as identifying those targeted by an intervention and then asking them questions that answer those of the evaluation? It is risky to assume that a population group targeted by an intervention are a homogenous collection of people, and associated with this, that the tools for data collection are appropriate to all in the group?
Take, as a common example, a household survey tool. Often used in research with an ethics committee approval, rarely does such a tool meet culturally appropriate requirements in developing countries. How, for example, is informed consent given by an illiterate person to be surveyed according to a set survey form? What response might an official looking form conjure in a homeless young person wary of authority? How can informants participate in such a process as equals?
Some methods of data collection in community level project evaluations select informants according to a statistical dissection of their community. While this process may meet statistical requirements it may also create suspicion and division in some societies. Where knowledge is seen as power, to have non-recognised persons providing knowledge which supposedly represents the whole community can potentially result in problems for the community once the evaluation team has departed.
So, what are some options to overcome these ethical dilemmas around data collection? First, the tools for data collection should be culturally appropriate and user friendly. Self-selection by informants in a user friendly, participatory evaluation where there is an opportunity to engage all groups in a community can ensure that minority groups such as the aged and infirmed, and marginalised groups such as women and young people, have the opportunity to give their view to the evaluation. In working with vulnerable groups, such as homeless young people, self-selection is crucial in ensuring informants experience a sense of control in relation to the data collection process and are not intimidated or threatened into participation. Evaluators have an obligation to ensure that participation is voluntary and that participants are provided with various means in which to choose to refuse to participate which ensures that they understand they have options regarding participation (Sabar, 2008). These means of refusal might be to tick a box on a form, provide verbal refusal to someone other than the evaluator, or be given the opportunity to leave the room within a certain period of time, and whilst the evaluator is also not in the room.
At the same time, methodologies for collecting data may need to be flexible and responsive to the shifting vulnerabilities of some population groups. By this we mean that some groups may be more vulnerable at some times throughout an evaluation process, than at others. For example, think about a newly homeless young person. Their sense of vulnerability may be higher than that of homeless young people who are (unfortunately) accustomed to the situation. In an international development context, asking young women in a village to participate in a data collection process around specific issues may potentially marginalise them through compromising their cultural gender roles. We cant assume that data can be collected using the same methods within various cultures, and sub-cultures in these cultures (Russ-Eft, 2004).
Further, rigorous adherence to a predetermined methodology may not allow participation. Other key considerations in addressing this issue are the building of trust and respect with informants and the time in which this can reasonably occur. If the time isnt available, do we simply press on and attempt to collect our data regardless of these considerations? In situations such as those just discussed, evaluators need to consider potential ethical issues in relation to data collection as early as possible in the planning of the evaluation methodology. This would include finding out as much about the informants prior to any data collection commencing and, if possible, working with them to ascertain a number of ways in which data can be collected appropriately and ethically.
Scenario two: the rights of the informants in accessing and reviewing the evaluation findings.
Nicholas has been commissioned by an NGO to evaluate a community development project that took place in a village in the Solomon Islands. He is only going to be in the village for four days before flying out. In that time he will need to interview all of the villagers and gather their stories about the project. Nicholas decides he will use some focus group-type of methods to gather the data as well as undertake a few individual interviews. Nicholas is concerned that he wont be able to check the accuracy of his interpretation of the findings of the focus groups and interviews with the villagers, but there is no time for this to occur. Nicholas has faith in his ability to accurately portray what informants have told him; he is very respectful of informants and would never misrepresent their views and opinions.
In the field of international development, community level projects are mostly implemented by non government organisations (NGOs). These NGOs regularly engage independent evaluators to train and lead evaluation teams to conduct end of project evaluations of their projects as a requirement of their contractual arrangements with the projects donor. While the independent evaluators are expected to act ethically and most NGOs adhere to codes of conduct as required by national umbrella groups and by donors, rarely are the terms of reference (ToR) for an evaluation subjected to ethical scrutiny by an independent ethics body. This places an added responsibility on the independent evaluator to ensure that all stakeholders interests are ethically upheld which may at times require decision making in the field without the of consultation of peers. This may be even more difficult whilst in the field and trying to ensure we involve informants in accessing and reviewing the evaluation findings. So how do we do this in an ethically responsible way?
Target communities in community development projects conducted in both the international and developed country context may be seen as partners at best, but most often are seen as beneficiaries which can sometimes mean that they are perceived as being in need of assistance by others who know better. In project evaluations it is the members of these communities who, as informants to the evaluation, provide vital information which informs evaluation findings and conclusions. It is also these community informants who are most often forgotten when it comes to being made aware of the outcome of the evaluation. Because of this it is vital that the evaluation process provides some value to the informants during the evaluation.
One way to consider this issue is to think about what the informants gain from the evaluation. Evaluations should provide something of value for all those involved. This includes the project donor who, most often, is also the evaluation commissioner. It also includes the project implementing organisation which, in the case of community development projects, is very often a non-government organisation (NGO). The target community is also a critical group in grassroots evaluation.
One option to address this notion of gaining something from the evaluation is through providing opportunities for open and inclusive participation. This may range from involving the informants in the design of the data collection and analysis phase of the evaluation to training informants to undertake sections of the evaluation. Techniques such as these not only can address the power and control imbalance between evaluator and informant, but provide a chance for informants to obtain value from the evaluation process, whether that value be information, skills building or increased confidence. Making such approaches explicit in responses to evaluation briefs can help to educate those commissioning evaluations about the need to provide a meaningful return for informants in the course of an evaluation.
Whats in, whats out
The collection of data, knowing that all the collected data will not ultimately be used in determining the findings and conclusions of the evaluation, is also an area where evaluators need to question their practice in terms of their relationship with informants. Lets revisit the first scenario, where Anja is interviewing homeless young people:
Anja finally has undertaken some interviews with homeless young people. In analysing the data, she decides to cut much of what was said that didnt seem to relate to the evaluation. She puts together a report, using quotes from the informants, but these have been edited to better reflect the key evaluation messages. The evaluation report is launched by the NGO and a powerpoint presentation is used to show some of these compelling quotes to the audience, which includes youth worker and some of the homeless young people who participated in the evaluation. A few of the young people recognise their words in the quote, but remark to the youth worker that they have been taken out of context. Other young people are puzzled; their stories arent mentioned at all. The NGO is thrilled with the report, and Anja feels that she has captured the essence of the informants experiences accurately. But the young people are confused and disillusioned about why their words were changed, and why they werent told this would happen.
The scenarios above are all too common. We are often pressured for time within the evaluation process to get the data collection done. In the midst of data analysis we find gems of information whilst simultaneously discarding what doesnt appear to fit with our brief. We can become so preoccupied with this next stage in the evaluation process that we can forget our ethical obligation to our informants; that the data is their lived experience and that they have an ethical right to not only review how we have interpreted their experience, but what we have subsequently done with it.
So what can we do to ensure that informants retain some power and control regarding their lived experience? We can ensure that we inform them at the outset of what will be likely to occur with their data. If we clearly are not going to be able to use it all, we need to tell informants that this is a possibility. We can also ask them if they would like to review it and allow them to provide feedback or make changes; particularly if later on data is de-identified and informants may not be able to have input regarding their data. Evaluation projects that are submitted to ethics committees will often be compelled to provide these options. However, ensuring that informants have the right to review what they have provided, make comment on this, and even withdraw it if they choose, needs to be an ethical principle followed in all evaluations, not just those approved by ethics committees. There is an ethical implication here that informants consent to contribute to an evaluation may be an ongoing negotiation rather than a one-off discussion at the start of the data collection process (Sabar, 2008; Stake and Jegatheesan, 2008). It needs to be clear that we are not implying that informants dictate what is or is not included in the evaluation in regards to personal agendas or interests; rather it is the obligation of ensuring that informants have the opportunity to view and make comment on what has been done with the data they have provided. The evaluator still needs to be guided by professional standards and not the personal interests of stakeholders (Russ-Eft, 2004).
Conclusion
The options discussed in relation to the scenarios above reflect situations where sometimes the rights of the informant are forgotten. Perhaps there is a need to not only reflect upon guidelines (or lack of) relating to evaluation, but also about the culture created by evaluators in responding to and providing evaluations, often under tight timelines or other restrictions. Schwandt (2007b:402) reminds us that evaluators need to assess their own culture to determine whether attitudes and practices have impeded understanding of informants perspectives. This could be expanded to also include consideration about and inclusion of informants in relation to from whom data is collected, how this is done, and the rights of informants in accessing and reviewing evaluation findings. Hopefully, this paper is another opportunity for the evaluation profession to consider and discuss further the challenge to be ethical in evaluation, particularly in relation to informants rights.
References
Morris, M. & Jacobs, L.R. (2000) You got a problem with that? Exploring evaluators disagreements about ethics. Evaluation Review, 24(4): 384-406.
Munk, M.D. (2002) Gender, Marginalisation and Social Exclusion. Welfare Distribution Working Paper 14/a:2002. Denmark: The Danish National Institute of Social Research.
Russ-Eft, D. (2004) Ethics in a global world: an oxymoron? Evaluation and Program Planning, vol 27: 349-356.
Sabar (2008) Informed consent: an instrumental or deceptive principle in qualitative educational research. Advances in Program Evaluation, vol 12: 63-82.
Schwandt, T.A. (2007a) On the importance of revisiting the study of ethics in evaluation. Advances in Program Evaluation, 10: 117-127.
Schwandt, T.A. (2007b) Expanding the conversation on evaluation ethics. Evaluation and Program Planning, 30: 400-403.
Schweigert, F. (2007) The priority of justice: A framework approach to ethics in program evaluation. Evaluation and Program Planning, 30: 394-399.
Stake, R. & Jegatheesan, B. (2008) Access, a zone of comprehension, and intrusion. Advances in Program Evaluation, vol. 12: 1-13.
PAGE \* MERGEFORMAT 9
) * ; < g h ~ * + 3 4 9 > k ~kUC1C "h" CJ OJ PJ QJ aJ nH tH "hK S CJ OJ PJ QJ aJ nH tH +hK S hK S 5CJ OJ PJ QJ aJ nH tH %hK S 5CJ OJ PJ QJ aJ nH tH "hpH CJ OJ PJ QJ aJ nH tH /h 5 hpH 0J 5CJ OJ PJ QJ aJ nH tH .j hpH 5CJ OJ PJ QJ UaJ nH tH %hpH 5CJ OJ PJ QJ aJ nH tH +hpH hpH 5CJ OJ PJ QJ aJ nH tH +hpH hK S 5CJ OJ PJ QJ aJ nH tH * + 4 D E C P t
$dh a$gd
( $dh d d [$\$a$gd^m $hdh d d [$\$^ha$gd^m $hdh d d [$\$^ha$gd $hdh d d [$\$^ha$gd= $hdh d d [$\$^ha$gdK S $hdh d d [$\$^ha$gd' k x |
#
$
0
u
' 3 F Q ] q { 6
;
e
j
ܸܸܸܸܸܦ܂ppʂʂpppd hv CJ OJ QJ aJ "hl CJ OJ PJ QJ aJ nH tH "h CJ OJ PJ QJ aJ nH tH "hpH CJ OJ PJ QJ aJ nH tH "h$Y CJ OJ PJ QJ aJ nH tH "hAkp CJ OJ PJ QJ aJ nH tH "h= CJ OJ PJ QJ aJ nH tH "hK S CJ OJ PJ QJ aJ nH tH "h CJ OJ PJ QJ aJ nH tH #
8 C D E ` B C O P ʻ⩗xk_S_G_ hl CJ OJ QJ aJ h" CJ OJ QJ aJ h_ q CJ OJ QJ aJ hOP( 5CJ OJ QJ aJ hK S 5CJ OJ QJ aJ "hBD CJ OJ PJ QJ aJ nH tH "h CJ OJ PJ QJ aJ nH tH "hl CJ OJ PJ QJ aJ nH tH h[Z h^m CJ OJ QJ aJ h^m CJ OJ QJ aJ h CJ OJ QJ aJ "h CJ OJ PJ QJ aJ nH tH h) CJ OJ QJ aJ % I ; < C _ m z
c n o } ĸБ܅ymymymymy[y "hl CJ OJ PJ QJ aJ nH tH hK S CJ OJ QJ aJ h CJ OJ QJ aJ h0 CJ OJ QJ aJ h,) CJ OJ QJ aJ h[Z h[Z CJ OJ QJ aJ hv CJ OJ QJ aJ h[Z CJ OJ QJ aJ hl CJ OJ QJ aJ h% CJ OJ QJ aJ h" CJ OJ QJ aJ h_ q CJ OJ QJ aJ hAkp CJ OJ QJ aJ P < H # $ M% & & >' ' * - / 1 3 q8 ;
$dh a$gd $
& F dh d d [$\$a$gdv
$dh a$gd
$dh a$gdK S $dh d d [$\$a$gd
( | ( 7 1 ~ * . 8 < F Q U z { ʾ֚vjjjjj^ h CJ OJ QJ aJ h,) CJ OJ QJ aJ hd CJ OJ QJ aJ h. CJ OJ QJ aJ h" CJ OJ QJ aJ h] CJ OJ QJ aJ h_ q CJ OJ QJ aJ h CJ OJ QJ aJ hpH CJ OJ QJ aJ hD CJ OJ QJ aJ hX8 CJ OJ QJ aJ h h 5CJ OJ QJ aJ h
( 5CJ OJ QJ aJ $ | L 4 > S f ˿˳˧˿ˏw˃kkk_S hv CJ OJ QJ aJ h; CJ OJ QJ aJ h CJ OJ QJ aJ h| CJ OJ QJ aJ hd CJ OJ QJ aJ h} CJ OJ QJ aJ hD CJ OJ QJ aJ h{1T CJ OJ QJ aJ h) CJ OJ QJ aJ h CJ OJ QJ aJ h:a CJ OJ QJ aJ hX8 CJ OJ QJ aJ h CJ OJ QJ aJ h h 6CJ OJ QJ aJ f x! H" " # # # $ $ % /% E% L% M% d% g% % % & 9& l& & & =' >' ĸznnzbzM (hv hv CJ OJ PJ QJ aJ nH tH hBD CJ OJ QJ aJ hG# CJ OJ QJ aJ hv CJ OJ QJ aJ hg CJ OJ QJ aJ hv 5CJ OJ QJ aJ hv% 5CJ OJ QJ aJ hv% CJ OJ QJ aJ h?$ CJ OJ QJ aJ h CJ OJ QJ aJ h; CJ OJ QJ aJ h<1 CJ OJ QJ aJ hv CJ OJ QJ aJ he/ CJ OJ QJ aJ >' d' ' ' ' ( ( ) A) * -* * * x+ + + , , , , X- Y- - - c. . / / / / /
0 )0 [0 ~0 0 0 0 F1 O1 L4 ϼthh h\S CJ OJ QJ aJ hz CJ OJ QJ aJ h CJ OJ QJ aJ hye CJ OJ QJ aJ hJ CJ OJ QJ aJ h CJ OJ QJ aJ hv CJ OJ QJ aJ hu? hz] OJ QJ hbB OJ QJ hu? h OJ QJ h87 OJ QJ hv >*䴳 ϴ ; ; >*䴳 ϴ (4 4 5 86 6 8 9 9 : : ; ; ; < < < > > > ? ? j? ? ? JB KB tB uB D 記sgsgs[O[ h; CJ OJ QJ aJ h
CJ OJ QJ aJ h87 CJ OJ QJ aJ hu? h CJ OJ QJ aJ +h87 h87 >*CJ OJ PJ QJ aJ nH tH h87 h87 >*CJ OJ QJ aJ h87 h
>*CJ OJ QJ aJ h CJ OJ QJ aJ hn:7 CJ OJ QJ aJ h^m CJ OJ QJ aJ h[ CJ OJ QJ aJ h CJ OJ QJ aJ h\S CJ OJ QJ aJ ; > ? KB iF GI +K UN kN O S V H\ S\ )` 4` ` pa a wb b tc d
$dh a$gdD
$dh a$gdv
$dh a$gd
$dh d d [$\$a$gd87
$dh a$gd87 D !E "E .E 0E yE E E 2F 3F ?F GF hF iF F F F F F G G )G G G G G G G HH KH H FI GI J J +K EK uK K K ĸи|ppp|d|| h^m CJ OJ QJ aJ hv% CJ OJ QJ aJ hor CJ OJ QJ aJ hg CJ OJ QJ aJ hH CJ OJ QJ aJ hv CJ OJ QJ aJ h
CJ OJ QJ aJ hz] CJ OJ QJ aJ h w CJ OJ QJ aJ h87 CJ OJ QJ aJ h}N CJ OJ QJ aJ h CJ OJ QJ aJ hVu; CJ OJ QJ aJ 'K M M TN UN kN N N O O rO O O S T 0T V lW W W )X Y Y Y ]Z jZ Z G\ H\ R\ ui]P h^m 5CJ OJ QJ aJ hn:7 CJ OJ QJ aJ h< CJ OJ QJ aJ h\j CJ OJ QJ aJ hz CJ OJ QJ aJ h CJ OJ QJ aJ h
( CJ OJ QJ aJ h87 h87 CJ OJ QJ aJ h87 CJ OJ QJ aJ h)n CJ OJ QJ aJ hg hg >*䴳 ϴ z 䴳 ϴ 䴳 ϴ ǰ 䴳 ϴ \ ( ) 3 4 ` ` ` ` ` ` ` ` Dz 貹 a a ˻wڰɰ? :7 :7 6C ϴ :7 䴳 ϴ <1 h<1 6CJ OJ QJ aJ h<1 CJ OJ QJ aJ he/ CJ OJ QJ aJ h,) CJ OJ QJ aJ hO& hO& 6CJ OJ QJ aJ hO& CJ OJ QJ aJ hO& hH 5CJ OJ QJ aJ hO& hO& 5CJ OJ QJ aJ hv% CJ OJ QJ aJ hz CJ OJ QJ aJ h^m CJ OJ QJ aJ h^m hg 5CJ OJ QJ aJ a Hb fb wb b b b b b Fc ec tc c c d Zd xd d d d d d d d d d d d d ؼ̬̠thYQMQMQMQM hDb j hDb U hv h{1T CJ OJ QJ aJ h{1T CJ OJ QJ aJ h^m h^m 6CJ OJ QJ aJ h^m CJ OJ QJ aJ hAkp hD 6CJ OJ QJ aJ hD CJ OJ QJ aJ hAkp hAkp 6CJ OJ QJ aJ hO& hO& 6CJ OJ QJ aJ hAkp CJ OJ QJ aJ hO& CJ OJ QJ aJ h< h< 6CJ OJ QJ aJ h< CJ OJ QJ aJ d d d d d d d d d d d d d d d d d $a$ d gdPJp
$dh a$gdv d d d d d d d d d d d hv h{1T CJ OJ QJ aJ hDb h rd hPJp j h<1 UmH nH u hz mH nH uh<1 j h<1 U
2 1h:pOP( . A!"#$% j 6 6 6 6 6 6 6 6 6 v v v v v v v v v 6 6 6 6 6 6 > 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 h H 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 2 0 @ P ` p 2 ( 0 @ P ` p 0 @ P ` p 0 @ P ` p 0 @ P ` p 0 @ P ` p 0 @ P ` p 8 X V ~ OJ PJ QJ _HmH nH sH tH J ` J OP( N o r m a l d CJ _HaJ mH sH tH D A D
D e f a u l t P a r a g r a p h F o n t R i R 0 T a b l e N o r m a l 4
l 4 a ( k (
0 N o L i s t < < PJp 0 H e a d e r
B# mH sH : /: PJp 0 H e a d e r C h a r CJ aJ tH < ` < PJp 0 F o o t e r
B# mH sH : /!: PJp 0 F o o t e r C h a r CJ aJ tH V 2V
PJp 0 B a l l o o n T e x t d CJ OJ QJ aJ mH sH R /AR PJp 0 B a l l o o n T e x t C h a r CJ OJ QJ ^J aJ tH 6 U`Q6 pH 0 H y p e r l i n k >*B* ph PK ! [Content_Types].xmlj0Eжr(Iw},-j4 wP-t#bΙ{UTU^hd}㨫)*1P' ^W0)T9<l#$yi} ;~@(Hu* Dנz/0ǰ $ X3aZ,D0j~3߶b~i>3\`?/[G\!-Rk.sԻ..a濭? PK ! ֧ 6 _rels/.relsj0 }Q%v/C/} (h"O
= C?hv=Ʌ%[xp{۵_Pѣ<1H0ORBdJE4b$q_ 6LR7`0̞O,En7Lib/Se е PK ! ky theme/theme/themeManager.xml M
@}w7c(Ebˮ CAǠҟ7՛K
Y,
e.|,H ,lxɴIsQ}#Ր ֵ+!,^$j=GW)E+&
8 PK ! P theme/theme/theme1.xmlYOo6w toc'vuر-Mn iP@I}úama[إ4:l ЯGRX^6؊>$ !)O^rC$y@ /yH*)UDb`}"qۋJחX^)I`nEp)liV[]1M<OP6r=zgbIguSe bORD۫qu gZo~ٺlAplxpT0+[}`j zA V2Fi@qv֬5\|ʜ̭NleXd sjcs7f
W+Ն7` g ȘJj|h(KD-
dXiJ؇(x$( :;˹! I_TS 1?E ??ZBΪmU/ ?~xY'y5g&/ɋ>GMGeD3Vq%'#q$8K )fw 9:ĵ
x}rxwr:\TZaG*y8IjbRc|XŻǿI
u3KGnD 1NIBs
RuK>V.E L+M2#'fi ~V vl{u8z H
* :( W ☕
~JTe\O*tHGHY }KNP*ݾ˦ TѼ9/#A7qZ $*c?qUnwN%Oi4 =3ڗP
1Pm \\9Mؓ2aD];Yt\[x]}Wr|]g-
eW
)6-rC Sj
id DЇAΜIqbJ#x꺃 6k#ASh&ʌt(Q% p%m&]caS l=X\P1Mh9MVdDAaVB[݈fJíP|8 քAV^f
Hn- "d>znNJ ة>b&2vKyϼD:,AGm\nziÙ .uχYC6OMf3or$5N HT[XF64T,ќM0E)`#5XY`פ ;%1U٥m;R>QD DcpU'&LE/pm%]8firS4d 7y\`JnίI R 3U~7+#m qBiDi*L69mY&iHE=(K&N!V.KeLDĕ{D vEꦚdeNƟe(MN9ߜR6&3(a/DUz<{ˊYȳV)9Z[4^n5!J?Q3eBoCM m<.vp IYfZY_p[=al-Y}Nc͙ŋ4vfavl'SA8|*u{-ߟ0%M07% < ҍ PK !
ѐ ' theme/theme/_rels/themeManager.xml.relsM
0wooӺ&݈Э5
6?$Q
,.aic21h:qm@RN;d`o7gK(M&$R(.1r'JЊT8V"AȻHu}|$b{ P8g/ ]QAsم(#L[ PK- ! [Content_Types].xmlPK- ! ֧ 6 + _rels/.relsPK- ! ky theme/theme/themeManager.xmlPK- ! P theme/theme/theme1.xmlPK- !
ѐ ' theme/theme/_rels/themeManager.xml.relsPK ]
\ * * * - k
f >' 4 D K a d d 3 5 6 7 9 : ; < = ? @ A B D P ; d d 4 8 > C ; g ~ \ X $ & - ! 8 @ 0 (
B
S ?
Y c l p )" -" m/ r/ 3 3 m< p< G G G G 8I BI J J ]R bR tR R ">$ CzmJ">$ R$Kf) h
^`h H . h
^`h H . h
pL^p`Lh H . h
@ ^@ `h H . h
^`h H . h
L^`Lh H . h
^`h H . h
^`h H . h
PL^P`Lh H . h
^`h H . h
^`h H . h
pL^p`Lh H . h
@ ^@ `h H . h
^`h H . h
L^`Lh H . h
^`h H . h
^`h H . h
PL^P`Lh H . h
8^8`h H . h
^`h H . h
L^ `Lh H . h
^ `h H . h
x^x`h H . h
HL^H`Lh H . h
^`h H . h
^`h H . h
L^`Lh H . h
8^8`h H . h
^`h H . h
L^ `Lh H . h
^ `h H . h
x^x`h H . h
HL^H`Lh H . h
^`h H . h
^`h H . h
L^`Lh H . h
^`h H . h
^`h H . h
pL^p`Lh H . h
@ ^@ `h H . h
^`h H . h
L^`Lh H . h
^`h H . h
^`h H . h
PL^P`Lh H . Q|> CzmJ R$K ]v- J a ` + 0 v
f " G# $ v% O&
( OP( + '- . <1 n:7 Vu; BD pH HK K S \S {1T i\ z] Db rd ? @ A B C D E G H I J K L M N O P Q R S T U V W X Y Z [ \ ^ _ ` a b c d f g h i j k l o p s R o o t E n t r y F .bKr 1 T a b l e F - W o r d D o c u m e n t 4 S u m m a r y I n f o r m a t i o n ( ] D o c u m e n t S u m m a r y I n f o r m a t i o n 8 e M s o D a t a S t o r e bKbK E L J D P N 4 P N R L A = = 2 bKbK I t e m
P r o p e r t i e s U C o m p O b j
y
F' Microsoft Office Word 97-2003 Document
MSWordDoc Word.Document.8 9q