AICAFMHA:
promoting mental health for young Australians
Australian Infant, Child, Adolescent and Family Mental Health Association Ltd
ABN 87 093 479 022
Consumer and Client Measurement Systems in the New Millennium: Fact,
Fiction, Utility and Implementation.
From: "The Common Data Base Project" Princess Margaret Hospital for Children,
Perth, WA.
Dr Peter Gibbon, T.R.E.E., University Department of Paediatrics, Princess Margaret
Hospital for Children
Measurement Systems
Database Project
Potential Problems of a measurement system
Benefits of a measurement system
Integration
Future directions
Conclusions
Consumer and client measurement systems in the wider context
The need for research and evaluation in clinical settings, including consumer and client measurement systems, is becoming ever more apparent. Some quotations from the 3rd National Getting Together Conference, "Measurement, Money and Mental Health in the New Millennium", held in Sydney in February 2000 provide an indication of why this is so -
- "In the past the equity argument was the main argument for mental health services. Increasingly the argument is becoming economic around investing in human and social capital." (Harvey Whiteford)
- Disability adjusted life years (DALY) methodology is now used by WHO and the World Bank. In terms of DALYs lost:
- 1990 - 10.5% of burden from Neuropsychological Disorders
- 2020 - projected to be 15%
- Burden is more severe in the West, accounting for 50% of total burden (WHO)
- In Australia mental health is number three in terms of burden of disease (cardiovascular, neoplasm, mental health) (Gavin Andrews)
- There is a trend for mental health resources to become increasingly scarce:
- Inpatient beds in US have dropped from 400,000 to 80,000 over last 2 decades
- In US managed care organisations money spent on mental health per member has decreased from $10.00 (1990) to 90c (2000)
- Medicaid has also lost 25-40% of funding (Ronald Manderscheid)
In Australia:
- The imbalance of funding versus burden is increasing
- There is a need to manage scarce resources
- Mental health spending in Australia = 0.4% GDP (Canada, UK, NZ ~ 0.8%) (Gavin Andrews)
In short, the combined effects of an increasing incidence of mental illness and
decreasing resources require that maximum benefit must be obtained from all mental health
interventions, and child and adolescent mental health is no exception. As a consequence,
the need for research that is designed to identify best practice strategies and for
evaluation to ensure quality of outcomes is increasingly apparent. In addition, there is a
growing movement towards increased consumer participation, including the measurement of
consumer satisfaction, on the basis of equity and inclusiveness. The task for researchers
working in clinical settings is therefore to provide quality research, evaluation and
consumer measurement, whilst producing outcomes which meet the needs of all stakeholder
groups. This paper identifies stakeholder groups and their expectations, details the
benefits of research based on client and consumer measurement systems in clinical settings
and offers some practical suggestions for implementing and maintaining such a research
programme. It is based on the six-year experience of the Common Database Project, which is
conducted within the Psychological Medicine Clinical Care Unit at Princess Margaret
Hospital for Children, Perth.
Back to top
Common Database Project
The Common Database Project of the Psychological Medicine Clinical Care Unit at Princess Margaret Hospital for Children commenced in 1996. Since that time it has formed a continuous case register of over 2,600 individuals and their families. All clients of the PsMCCU are invited to participate, and the current participation rate is over 75%. Of the clients registered on the database 55.2% are male and 44.8% female. Their ages range between 4 and 18 years (M = 10.68, SD = 4.07). Since its inception, the Common Database Project has evolved and its processes have been refined in order to ensure its continued relevance to clinical practice, to maintain high consumer compliance rates, and to keep abreast of contemporary standards of ethics and confidentiality.
Possible Stakeholders
There are a number of stakeholder groups in the domain of child and adolescent mental
health services. These include clients, their parents, clinicians, other staff and
management of the host organisation, funding providers and researchers. Each of these
groups will possess its own agenda and its own concerns regarding the relative importance
of aspects of any consumer based measurement system. Successful research and evaluation
based on consumer measurement therefore must address the concerns of all stakeholders, and
provide each with some tangible, readily recognisable benefit.
Back to top
Potential problems with a measurement system
Each stakeholder group will perceive potential problems, specific to their interests, associated with the implementation of a consumer based measurement system. Clients and their parents are likely to have concerns about the relevance of the research endeavour to the individual’s clinical care, the confidentially of their personal information, and the time burden imposed on them by the need to comply with data collection procedures. Parents may also be concerned about possible misinterpretation of results and that research findings will be employed in a process of parent blame or to imply parental abuse.
Amongst staff potential concerns will include the burdens of time and money, any support for research being viewed as a diversion of resources from clinical services. "Research agenda suspicion" is also likely among non research staff, that is, the belief that research is conducted only to further academic careers or as grist for the conference circuit mill. Clinical staff could also perceive research which is not directly client focussed to be of little benefit to individual clients and may question the validity of coalface research, particularly if they perceive research models to be too simplistic to address the complexity of the problems under investigation. Delays in making clinically relevant data available to clinicians within a viable time frame can also be a cause for concern among staff.
For management of the host organisation, there will be issues surrounding funding and the management of large case registers and databases. They will have to ensure that ethical standards are adhered to at all times in the data collection and storage procedures, for instance ensuring that properly informed consent is obtained from all clients. Management will also have the added responsibility of ensuring that data are only released in de-identified format to legitimate end users via a formalised and documented data accessing procedure.
Benefits of a measurement system
In contrast to the acknowledged difficulties in operating a consumer based research and evaluation programme within a clinical setting, there are also potential benefits for each of the stakeholder groups. Clients and their parents will benefit from any direct impacts on patient care, and will be reassured by the perception that research and development occurs in progressive, dynamic organisations. Consumer based measurement systems also allow consumers to have input into the treatment process and to state their satisfaction or otherwise with aspects of the service being provided.
Benefits for staff include a broadening of the range of tasks which they are required to perform, bringing with it an expansion of their skills base and diversity in their work environment, potential higher education opportunities, and the professional stimulus of working in a milieu of curiosity and investigation.
For management, an ongoing programme of consumer measurement addresses the ethical
requirement to evaluate treatment and to engage in a process of quality improvement and
offers protection of services and resources via good data and outcome measures. It can
also provide evidence over time which confirms clinician anecdotal experience, identifies
trends and feeds into new and existing service provision. Research outcomes that are
publicly disseminated via conference presentations and publication in professional
journals also raise the profile and status of the organisation.
Back to top
How to integrate measurement systems into clinical organisations
A number of organisational models are possible for integrating consumer measurement based research and evaluation into clinical settings. These include:
The grid/matrix model. This model is relevant to those organisations in which specialised units or programmes offer specific services. Under this model an independent research and evaluation unit operates across all functional units in the organisation, ensuring uniformity of data collection, research and evaluation procedures.
The service provider model. This model also requires a stand alone research and evaluation unit. Under this model the unit provides research services to the organisation as required.
The clinical leader model. Under this model the clinical leader directs the research and evaluation endeavour. This model works best when the clinical leader within the organisation also possesses research expertise.
The external consultant model. An alternative to an in house research program is to contract external research providers, either on an ongoing or project specific basis. This model is particularly appropriate if it is necessary to obtain an independent evaluation of a service.
The "by higher degree" model. It may be appropriate and effective in some settings to have research undertaken by postgraduate students assigned to the organisation for clinical and/or research supervision. Specific research projects that are required by the organisation can form the basis of quality applied research theses, a situation from which the organisation and the student benefit.
The choice of model will depend on the characteristics of the host organisation,
expectations of the research programme, funding and resources. It may also be necessary to
have a mix, for instance creating an in house research unit which conducts applied
research, but employing an independent body to conduct evaluation projects.
Back to top
Getting it done, the here and now of integrated measurement systems
A number of important issues must be addressed if the integration of a consumer measurement system into a clinical setting is to be successfully achieved. A formal structure governing the implementation and maintenance of the data collection procedure is vital. The purpose of the data collection, description of the collection procedures, designated responsibility for data management, appropriate uses for the data and a procedure for requesting access to the data should all be documented in a research charter. Additional documentation such as Ethics Committee approvals, request forms and the data collection instruments themselves should be appended to the charter to form a complete set of research protocols. Regular reporting on the research programme, for instance in an annual report format, should be initiated as a core component of the programme.
In addition to the formal procedures which should be implemented, a number of practical strategies should also be put in place. These include developing a sense of ownership across the organisation, so that all internal stakeholders, from management and clinicians through to clerical staff, are invested in the research programme. The presence of a dedicated leader and administrative support, including professional data entry and management is also necessary. The consumer measurement system itself must be user friendly and encourage compliance among consumers. This can be achieved by implementing strategies which facilitate consumer responsiveness, such as personalised letters inviting participation, mailouts of data collection instruments which allow clients to complete them prior to attending the clinical service, and having trained clerical staff available to assist clients with any queries that they may have.
The future of client and consumer based measurement systems in child and adolescent mental health will be shaped increasingly by the requirements of governmental agencies and fund providers. A nationwide measurement system has recently been proposed (Bickman, Nurcombe, Townsend, Belle, Schut & carver, 1999). Whilst this may necessitate reconsideration of current consumer measurement systems such as the PMH Common Database project, it offers the opportunity to make meaningful comparisons between alternative measurement strategies. For instance, it offers the opportunity to compare current research programmes across different sites and to compare clinician completed and client based measures.
Ultimately, a national consumer measurement database will facilitate longitudinal
studies, which will greatly enhance programme and treatment evaluations and make a major
contribution to clinical practice feedback. This will be particularly true if consumer
data can be integrated with occasions of service data, greatly enhancing the scope of
potential research.
Back to top
Client and consumer measurement systems are increasingly a necessary part of the way in which child and adolescent mental health services do business and should be an integral component of research and evaluation programmes. They provide meaningful data from the client stakeholder group and can provide valuable evidence of the efficacy of services provided. To be successful, such measurement systems must be structured to meet the needs of all stakeholders, to fit easily into the business of the host organisation, and to be user friendly for consumers.
Printable Version