Advent Consulting Associates
3541 Pacifica Lane
Elk Grove, CA 95758
ph: 916-753-3993
jerry
Truly effective performance management and program evaluation efforts center on four general steps depicted in the following figure and described below.
Basic Evaluation Methodology of Advent Consulting Associates
Our application of research skills begins with an understanding of the question(s) the client seeks to answer. We make special efforts to understand the client’s informational needs and the long-range goal of the evaluation findings. For example, some evaluations might be for purposes of monitoring and updating progress of on-going programs, while other studies are conducted to guide important decisions about funding, structure, and operations. The ultimate purpose of each study helps to determine the scope of the effort (e.g., thoroughness and level of detail needed) as well as its general design and cost. When necessary, we are prepared to conduct a thorough search of published research as a foundation for focusing investigation into well-defined issues.
Our professionals apply research design principles to suit the particular needs, resources, and goals of the evaluation. We help clients and stakeholders to clarify expectations of performance and outcomes, identify measureable criteria and key indicators of success and effectiveness, and a variety of resource requirements for performance improvement.
Two aspects of research are especially important in evaluation: a) the identification of clear and explicit criteria; and b) appropriate methods for measuring those criteria. It is largely with the criteria selected that we can evaluate performance (of a person, program, or policy), and we need an effective and reliable measuring process for any criterion measures to be valid.
Our experience has shown that many programs and policies are introduced and implemented without a lot of detailed planning for evaluation. In these situations, criteria might have to be constructed so that they accurately reflect the original purpose and nature of the program or policy, such as by studying initial legislation and agency charters or by interviewing subject matter experts. We highly recommend formalizing a consensus by the client and other stakeholders about the validity and adequacy of evaluation criteria as early in the process as possible in order to ensure acceptance of the evaluation results.
Measurement methodologies must be appropriate for the types of data that are available and selected for criteria. On the whole, we prefer objective measures over subjective ones for complex evaluations, although we know that both types of data have important value. Sometimes, careful analyses of scales of measurement, sampling procedures, and normative references (e.g., comparison groups, historical records) are needed to assist in interpretation of complex processes where numerous outcomes occur over different times. In most evaluations of complex programs and activities, the measurement of a multiple criteria is required in order to obtain a full picture of a program’s operations.
In special circumstances, we can tailor research approaches and analytical techniques from a variety of research disciplines, including basis statistical process control, quality improvement techniques, and organizational assessment. We also offer an array of reporting and feedback processes to help clients assimilate and apply the lessons learned from evaluation findings.
We employ a variety of techniques for the collection of relevant data, tailored to suit the nature of the investigation. These include procedures designed for collecting both quantitative data and qualitative data such as the following:
Advent Consulting Associates applies both quantitative and qualitative methods to analyze data from research, and identify scales of measurement that are appropriate for the specific sources and types of data. We often perform descriptive statistics of the sample(s) and relevant stratification groups, and statistical tests of research hypotheses. Our consultants are very familiar with parametric, non-parametric, multivariate, and univariate statistics and their interpretation.
Our consultants are skilled in the communication and writing of research results, from a description of the research background through the interpretation of findings for decision-making by policy experts, elected officials, and agency managers. We can assist in the preparation of materials for public outreach and presentations, such as slides and graphs, tailored to the characteristics of each specific audience. We are familiar with a variety of formal writing styles including APA, economics, and Turabian styles.
Our approach to performance management is similar to our program evaluation research, at least conceptually. Both processes focus on acquiring and analyzing data, comparing results with expectations, and being fair and impartial. An important difference is that performance management is usually focused on individual performance and it is much more concrete than is the evaluation of programs, operations, policies, and organizations. Because of interpersonal difficulties, stress, and uncertainty, many managers find that performance management is an exceedingly difficult responsibility.
The basic elements of performance management for an individual employee require communication and fairness of treatment. The basic steps of the process include the following:
At the individual level, remaining objective is more difficult for a supervisor or manager than when focusing on the effectiveness of a program. It is both more personal and more stressful to deliver job-relevant feedback that is perceived as threatening to a fellow employee. In addition to the potential threats to an employee’s employment (e.g., security, income, benefits coverage), risks for supervisors also can arise from the process. On occasion, the application of performance management steps with individual employees has resulted in legal and financial liability for the managers involved.
We have found that the complex emotional issues entailed in performance management require careful planning and considerable preparation on the part of managers in order to carry out those responsibilities effectively.
With technical and consultancy experts who have worked as business and public service leaders themselves, Advent Consulting understands first-hand the challenges our clients face:
As a partner in performance management and program evaluation, Advent does not bring cookie-cutter solutions. Each organization is unique and deserves a customized, client-centered strategy for assessment, analysis, planning, and action.
We offer extensive experience in the application of research methodologies and statistical analysis techniques for performance management and program evaluation studies. Basic features of our approach are the close engagement of our clients, or stakeholders, coupled with flexibility in our methods to ensure our assessments meet the unique needs and situation of the organizations we serve.
Our methodologies emphasize the following as key characteristics for performance management and evaluation efforts:
We offer a track record of success in these types of initiatives.
Copyright Advent Consulting Associates, Inc. All rights reserved.
Advent Consulting Associates
3541 Pacifica Lane
Elk Grove, CA 95758
ph: 916-753-3993
jerry