Begin main content

Policy Perspectives on the Obsolescence of Health Technologies in Canada

Download this document


The persistence of ineffective or harmful practices in mainstream health care is a worldwide problem. Research suggests that 20% to 25% of patients have treatments that are unnecessary or potentially harmful and 30% to 40% of patients do not receive treatments of proven effectiveness. Yet internationally there are limited systems in place to identify, reduce, and/or withdraw redundant, ineffective, or inappropriate health care practices. Such practices result in suboptimal care and the inefficient allocation of scarce health resources.

Within this context, disinvestment has been referred to as the process of withdrawing (partially or completely) health resources from those existing health care practices, procedures, technologies, and pharmaceuticals that are deemed to deliver no or low health gain and are thus not efficient health resource allocations; the explicit aim is to reallocate those resources to areas of greater clinical and cost-effectiveness. However, there may be important differences in effectiveness and/or cost-effectiveness for the same technology or procedure across patient subgroups, providers, or institutions. Thus, disinvestment efforts must acknowledge that few technologies and procedures will be candidates for complete disinvestment, but may be better suited to partial retraction.

Internationally, there are few examples of successful disinvestment as part of a formal policy agenda, which reflects in part the strength of political will required to support the agenda.

Both conceptual and practical challenges exist. These include defining the balance of evidence required; identifying and prioritizing technologies and practices of questionable benefit; developing appropriate research methods; structuring an appropriate assessment method; securing resources to support policy development; and, perhaps strongest of all, overcoming political, clinical, and social challenges to removing established (entrenched) practices.

Established methods from health technology assessment provide a sound base on which to advance this agenda, but they will require development to adequately address the complexities associated with assessing established (legacy) technologies and practices.

Disinvestment is possible through numerous avenues, and these are discussed. The relevance and appropriateness of each will vary depending on the health care context. The most compelling of these are represented by a variety of reimbursement levers, by which funding is restricted for ineffective or inappropriately applied technologies or practices, thus circumventing the problem of differential uptake of recommendations (guidelines).

In this document we conclude with a series of suggested approaches, including:

  1. High-level decision and commitment to make disinvestment an explicit, formal, and resourced policy agenda.
  2. A regulatory framework for disinvestment decision-making that is transparent and removed from vested interests.
  3. Either (a) additional resources for existing committees to consider existing items or (b) the establishment of new, parallel committee(s) to consider existing items.
  4. Regulatory support provided for health technology assessment recommendations for (a) removing or (b) reducing reimbursement or (c) restricting use of a technology or practice.
  5. A centralized process to systematically and transparently identify existing, potentially ineffective practices on which to prioritize candidates for assessment.
  6. Clear pathways of appropriate modification to reimbursement drivers.
  7. Dedicated stream of funding for capacity building in research and policy development in disinvestment.

In order to succeed, approaches to disinvestment will need to be based on strong collaborative partnerships that will support methodology development, provide a regulatory foundation, and ensure decisive policy action.

Back to top


Definition of Health Technology Obsolescence and Disinvestment

This discussion paper is intended as a companion piece to Joshi et al.3 wherein obsolescence is defined as the “end-point of all technology, which can progress through a lifecycle that encompasses ideas, innovation, invention, investigation, adoption, acceptance, reduced use, and obsolescence.” Joshi et al. go on to posit that “Canada...needs to engage in a systematic process to reassess health technologies with regards to reduced use or decommissioning [of technology deemed potentially obsolescent].”

This paper will broaden the discussion and expand on the definition provided by Joshi et al., to progress from the binary implications that the term “obsolescence” holds to a more nuanced definition that acknowledges that in many instances a technology or practice is not necessarily obsolete for all indications and all patient characteristics, but that there may be a degree of inappropriate use or overuse. Thus the conversation may rest less on eliminating and more on restricting or refining indications for (appropriate, reduced) use.

In the early 1990s, claims were made that in all areas of health care, “30–40% of patients do not receive treatments of proven effectiveness,”5 and “20–25% of patients have treatments that are unnecessary or potentially harmful.”6 As a result, the goal of reducing the use of ineffective technologies or practices has been central to evidence-based medicine for well over a decade. Internationally, considerable effort and resources have been invested in developing well-defined criteria and evidence-based policy processes for assessing new and emerging health technologies, surgical procedures, and pharmaceuticals to gauge their safety, effectiveness, and cost-effectiveness. Increasingly, reimbursement approval (and, therefore, universal access) for these new services rests on stringent health technology assessment (HTA) processes. Of growing concern however, is recognition that these stringent assessment methods are relatively novel, and that the processes to date, while focusing overwhelmingly on new and emerging practices, technologies, and pharmaceuticals, have largely ignored existing services. Thus, there exists a legacy whereby many currently implemented health care interventions were in use prior to well-defined standards of effectiveness becoming a criterion for reimbursement, resulting in many jurisdictions being “stuck with the old and overwhelmed by the new,”2 the latter point reflecting that current evaluation capacity is stretched to the limit. Additionally, procedures or technologies that initially met criteria for safety, effectiveness, and cost-effectiveness may suffer from “indication creep,” whereby the indications for their use are altered beyond those originally considered. This may result in the ineffective use of otherwise effective technologies, resulting in ineffective care and inappropriate resource allocation.

In this paper, analyses and discussion points will be provided, with accompanying case studies and insights taken from the activity often referred to as disinvestment 12,7:

Disinvestment relates to the processes of withdrawing (partially or completely) health resources from any existing health care practices, procedures, technologies and pharmaceuticals that are deemed to deliver no or low health gain for their cost, and are thus not efficient health resource allocations. Within this is the view to re-allocation/re-investment towards technologies, practices and programs with greater demonstrated (cost-) effectiveness.2

Despite growing political support and an increasing number of initiatives in this area, the term disinvestment* has been met by some with a lukewarm reception. Given “dis-” infers a negative or reversing force — to undo (an investment) — there is, for funders and policy decision-makers, an awareness that disinvestment may carry with it connotations of reductions in the funding of health care — a perennially sensitive issue. It is, therefore, important to stress that the concept aligns better with notions of displacement and reallocation. Can continued investment in health care occur without thoughtful, measured displacement and reallocation? Arguably there is an economic imperative to do so for the sake of sustainability. There is also an ethical imperative for delivering quality health care to patients and a best practice imperative for purchasers and providers of health care services.

It is also important to note that disinvestment differs from rationing in both concept and intent. Recent work has defined rationing as “withholding beneficial interventions for cost reasons.”8 Conceptually, disinvestment approaches the need to confine health care spending from a different view. Disinvestment holds that, by eliminating ineffective care, waste is concomitantly reduced, hence resources are saved. The focus of disinvestment is on reducing ineffective or inappropriately applied practices, thus improving care and reducing waste without the need to withhold effective care through rationing approaches.

Although there may be differences in nomenclature, the essence of the process under discussion is that (a) it does not entail an all-or-nothing approach but can occur in degrees and (b) refocuses on a positive aspiration — the reallocation of funding to interventions and programs to those groups of patients most likely to benefit. So, one way of drawing a boundary around the terminological proliferation is to highlight the economic concept of opportunity cost. Ultimately, our resources are scarce, and so whenever a less cost-effective health practice or technology continues to attract funding, some people experience ill health because a better intervention is unavailable to them. Disinvestment thus opens the way for health gain — from the better deployment of health resources.9,10

It may be that all recipients of a health care practice or technology receive a less than optimal benefit from the amount of resources available or, alternatively, that a situation of diminishing marginal returns may apply in which, as the indications for the use of the practice or technology are widened, the additional benefit gets less and less. Eventually, the marginal cost of receiving this diminishing marginal benefit may be more than society (through its decision-makers) is willing to pay for.

* The term disinvestment usually relates to decisions about capital, whereas what is argued for in this context is a reduction in the consumption of certain health care interventions, albeit accompanied by human capital and physical capital investments.


The current discussion paper will focus on medical procedures as well as health technologies. Examples involving pharmaceuticals will be used only for illustrative purposes.

We aim to stimulate discussion and encourage debate among Canadian policy-makers. We will:

  • Provide a short background and select history of developments in this area
  • Describe barriers and challenges related to decommissioning of health technology
  • Provide an assessment of barriers and challenges, including examples of technologies that are obsolete for some, but not all, indications.

We will propose policy approaches within the Canadian context including:

  • Potential strategies to address the obsolescence of health technologies in Canada, including how to plan for and carry out disinvestment, including facilitating reductions in inappropriate overuse
  • Building from the points above, we will describe possible approaches to this issue in a Canadian context.

Back to top


A Brief History

Perhaps the most potent single indicator of the need for disinvestment comes from the considerable work of Wennberg et al., as presented through The Dartmouth Atlas of Health Care project in the USA. The Dartmouth Atlas documents serious defects in the quality of care now provided in the fee-for-service medical system. One member of this collaboration has stated:

There is substantial overuse, under use, and misuse of medical care in the United States. Interventions that are of little value are commonly overused; care that is effective is commonly underused; and care that is of unproved value is frequently misused. Spending on medical interventions continues to increase without evidence that doing more results in better outcomes or better patient satisfaction.
(Wennberg, as quoted in Daniels and Ramey11 p.187)

Indeed, many patients consider the care they receive care to have little or no benefit, as recent work from The Commonwealth Fund, reported by Schoen et al.,12 demonstrates. A seven-country comparison of public perceptions relating to treatments of little or no benefit found that between 10% (the United Kingdom) and 20% (USA and Germany) of survey respondents perceived that in the last two years a doctor had recommended treatment they felt had little or no benefit. The figure was 12% in Canada, 13% in The Netherlands, 15% in New Zealand, and 17% in Australia.

Attempts to address the phenomenon have a long, fragmented history. The following brief examples from Canada, the USA, Australia, and the UK provide some context.


“Delisting” activities occurred within Canadian provincial health insurance plans through the 1990s. In Appendix 2 we provide tables of candidate technologies and practices targeted in that process. Giacomini13 provides a critique of these activities and reports how candidate services (see Appendix 2, Table 1) “vary profoundly in specificity and scope, as well as along other dimensions...” (p. 728). She goes on to suggest that the selection process, “illustrates the variety of principles and interests that can assemble technologies for judgement” (p. 728). The candidate technologies for delisting were nominated by the medical association and the ministry in question, then reviewed by “an ad hoc commission of individuals...” (p. 728). Giacomini reports that the process was opposed by powerful medical lobby groups, and some interventions (along specialty lines) were removed from consideration.

Giacomini is equally critical of the process for selection of physician services in Table 2 (Appendix 2), suggesting these “were handpicked according to principles that were not well articulated or publicized by the decision makers” (p. 730) and that “the chair of the commission that reviewed the services for de-listing describes the final assembly as ‘bizarre’” (p. 730). This outcome illustrates the need for the application of uniform guiding principles in any such process, a point we consider in more detail later in this document.

Within Canada, program budgeting and marginal analysis has provided one framework for multi-criteria decision analysis — as an interdisciplinary change management process rather than a purely economic technique.14,15 Program budgeting and marginal analysis activities in Canada have principally concentrated across programs at the macro level. The approach to disinvestment described in this document differs in that it starts with identifying and prioritizing individual technologies and practices to review for clinical and cost-effectiveness. Thus, disinvestment is not a rationing approach but an attempt to target and eliminate waste.

United States of America

Cotter, in the Health Affairs blog,16 provides a review of the circuitous policy processes that have occurred within the USA in relation to addressing these challenges. Cotter documents the US Department of Health and Human Services National Center for Health Care Technology (NCHCT), “whose charge was to assess the usefulness of established and new medical technologies.” In 1978, the NCHCT (having a $4 million budget and a staff of 20) was authorized by Section 309 of the Public Health Service Act to conduct and sponsor assessments of health care technologies. Cotter reports that the NCHCT was not regulatory and fulfilled two main functions:

  1. Multifaceted assessments of technologies (e.g., safety; efficacy; effectiveness; and cost-effectiveness, social, ethical, and economic impacts) and
  2. Scientific and medical evaluations for use in making Medicare coverage decisions (i.e., determining whether specific procedures are “reasonable and necessary” and thus appropriate for reimbursement by Medicare).

The NCHCT’s tenure was short-lived (1979-1982), and Cotter posits the reason as opposition principally from the Health Industry Manufacturers Association (now the Advanced Medical Technology Association) and the American Medical Association and the strong anti-regulatory sentiments that swept into Washington in the early 1980s with a change from a Democratic to a Republican administration.

Despite this history, the USA can claim some of the first attempts at disinvestment, dating from the Blue Cross/Blue Shield Medical Necessity Project of 1976, in which professional colleges worked with the insurer to identify “outmoded and useless procedures.”17 This project resulted in 76 tests being removed from the insurance coverage of Blue Cross/Blue Shield. Neumann et al.18 point out that US Medicare has within its remit the ability to accept a request (internal or external) for a national coverage decision for a technology that “poses possible benefit or harm to Medicare beneficiaries.” This suggests that disinvestment on the grounds of harms (with ineffective treatment arguably representing harm) could be undertaken by Medicare regardless of the current comparative effectiveness research agenda.

United Kingdom

The term disinvestment (within the health care context) was coined in the UK where it has been adopted by the National Health Service — through the National Institute for Health and Clinical Excellence (NICE) — as a formal policy entitled “optimal practice reviews.”19 The disinvestment agenda in the UK emerged in 2005 as one of four streams of system reform. The first three were prevention, system inefficiencies, and administrative waste. Disinvestment followed from the fourth: clinical waste — underuse, overuse, and misuse of services. The perspective sought to capitalize on HTA and NICE’s “technology appraisals.” Herein, NICE could search for ineffective services through literature reviews and input from practitioners. Following this, it was envisaged that NICE would make disinvestment an explicit part of its remit within guideline development. NICE guidelines now incorporate a monthly set of “Recommendation Reminders” with a cost statement indicating potential savings. In parallel, NICE “Commissioning Guides” provide commissioners at the primary care trust (PCT) level with qualitative and quantitative advice on the services they should be commissioning from providers, together with information on their current level of activity in comparison with other trusts. As PCTs are not compelled to follow NICE guidelines (for disinvestment), this approach has resulted in a degree of variability of uptake (of recommendations), which in turn has contributed to the phenomenon referred to as “postcode rationing” and exposed some of the limitations of guideline-based approaches to reform.

Undoubtedly NICE is placed in a middle position, with politicians on one side advocating for disinvestment outcomes and PCTs on the other who have discretion to adopt (or not) NICE disinvestment recommendations. A 2008 House of Commons Health Committee report19 expressed disappointment in a perceived lack of progress in achieving disinvestment within NICE. Debate has ensued as to whether or not its powers should be bolstered above that of “recommendations” to that of regulatory effect.


In Australia (at the national level), the assessment (comparative safety, comparative effectiveness, and cost-effectiveness) of existing health care practices is within the mandate of the body responsible, the Medical Services Advisory Committee (MSAC) (and, for pharmaceuticals, the Pharmaceutical Benefits Advisory Committee). The MSAC is positioned, therefore, to advise the Minister for Health of items to remove from government funding schedules (Medicare). Despite this potential capacity, Australia has little record of disinvestment on effectiveness grounds — as opposed to on safety grounds. Reasons for this include the inability of MSAC to respond to the volume of applications for new and emerging items to join the schedule of benefits for reimbursement. In addition, there is no process for nominating candidates for disinvestment assessment and lack of agreed frameworks for the burden of evidence required to support a decision. Developments are afoot, however, as in the most recent federal budget (May 2009),20 the government has set aside AUD$9.3 million to:

…put in place a new evidence based framework for reviewing services listed on the Medicare Benefits Schedule. The new framework will take effect from 1 January 2010.

Under the new arrangements, services will be evaluated and aligned with contemporary evidence to ensure clinical relevance and appropriate pricing. New services will be evaluated three years after being listed. This will improve health outcomes for patients and help maintain the financial sustainability of the Medicare Benefits Schedule.

Back to top


The preceding items have hinted at some of the difficulties encountered internationally when various authorities have tackled this issue. The next section will identify and address challenges and barriers systematically. In addition, we will provide some suggested approaches to overcome these. Importantly, some barriers — and their solutions — are highly context- and/or country-specific whereas others have universal relevance.

Achieving Universal Reform with Degrees of Decentralization

In many countries, health care administration occurs at both the national/federal level and at the state, province, or regional level. Decentralized structures create a challenge in the potential for differential coverage (differential implementation of disinvestment) between states, provinces, or regions. This phenomenon is faced by NICE (a centralized body) and its recommendations to PCTs (PCTs are decentralized) — which is discussed in section 2.1.3. Often federal governments and agencies cannot act to impose universal regulations without the agreement of each state or province. To overcome this, any disinvestment agenda would need to involve states and provinces in order to give consideration to local contexts. Multilateral agreements on the part of the provinces and the federal government could foster universal reform in this area.

Lack of Dedicated Resources by Key Stakeholders to Build and Support Disinvestment Policy Mechanisms

[In Australia] by and large, the focus has been on adverse events, safety issues and that has been a big enough challenge of the last decade. The key question that we’re talking about, they’re [MSAC*] absolutely inundated and overwhelmed with new and emerging, they have no capacity to look at this [existing items]. (S9)†
*The Medical Services Advisory Committee (MSAC) advises the Australian Minister for Health and Ageing on evidence for safety and (cost)-effectiveness of medical technologies and procedures. This advice informs decisions about public funding.
†Participant pseudonym(s) provided (e.g., S9 = Stakeholder 9).
From Elshaug et al.21

The bearer of financial risk for the cost of health care may have the greatest incentive to drive a displacement and reallocation agenda. Internationally, one challenge is shared by all funders; the burden for proving effectiveness lies with the sponsor of a new device or pharmaceutical. However, for potentially obsolete technology or practices, the opposite occurs. The regulator or payer first has to identify or be made aware of a doubtful practice, then has to commission reviews, and then has eventually to mount a compelling argument for ineffectiveness and/or cost-ineffectiveness in a situation where defining and proving inferiority is conceptually difficult. For illustrative purposes, this may be described as somewhat analogous to standards of law. That is, adding an item to a funding schedule is beholden to a “balance of probabilities” standard whereas removal of an item requires a standard of evidence that is “beyond reasonable doubt.”

Most HTA bodies and decision-making committees have a full agenda focusing on new and emerging technologies and hence have limited capacity to address existing or legacy services. The adoption of disinvestment as a formal (and funded) policy agenda in the UK, and similar provisions in Australia’s most recent federal budget, point to the importance of a shift in political priorities as necessary to foster policy-driven disinvestment capacity.

Lack of Reliable Administrative Mechanisms to Identify and Prioritize Technologies and Practices for which there is Relative Uncertainty as to Clinical and Cost-Effectiveness

Disinvestment may be easier with pharmaceuticals and/or when adverse events occur. The process is more complex when individuals are not harmed by existing practices but are inappropriately or ineffectively treated by being subjected to diagnosis or treatment that may be safe but is of little or no meaningful clinical benefit (based on clinical and cost-effectiveness evidence). A register for events demonstrating “ineffective treatment” does not exist in the same way as an adverse event register exists for pharmaceuticals or as an adverse event register exists within tertiary care settings.

Further compounding this issue is that the number of groups in Canada who hold a clear directive and sufficient resources to seek out, identify, and investigate potentially redundant or ineffective procedures is limited. Wilensky has recently intimated that such limitations exist in the USA.22 This reflects a view that substantial additional investment is required to support evidence-based review not only of new and emerging but also of existing health care technologies, including comparative studies examining new versus existing practices. In high income countries such as Canada, the incentives support diffusion and not retraction or disinvestment. Similarly, existing HTA review models are geared (and effectively so) toward controlling the tap as it is turned on, not toward neutralizing the flow through active disinvestment. It is interesting, for example, that old technologies or practices are not routinely formally decommissioned as new items are approved. Instead, the range of options (and reimbursement items) grows ever larger. Although it appears that some practices gradually decline, the question remains whether this represents a sound policy approach to resource allocation and clinical excellence in health care.

The Canadian Agency for Drugs and Technologies in Health, for example, provides advice on whether a new technology is as, or more, safe, effective, and cost-effective as a (funded) comparator. There is no obstacle to recommending the removal of one service in parallel with a recommendation to add a new service to the reimbursement schedule should the new service have demonstrably better cost-effectiveness for a given indication. That the comparator is not automatically removed from the schedule, or has limits placed on its use, highlights a challenge for a disinvestment program. The new service (even if superior) may take time to diffuse and become accepted by health care providers. Premature disinvestment of the comparator may disadvantage patients where the new service is not yet available and thereby raise issues of access and equity. The evidence supporting the new technology may not go as far as assessing the cost-effectiveness of disinvestment of its comparator (in terms of capital investment, training, and changes to clinical workflow). There are also challenges where the existing technology (i.e., the comparator) may have been introduced without formal assessment. Thus the new intervention is being compared with an existing item for which there is poor basis for funding in the first place. These challenges are surmountable; however, providing assessment processes and decision-making in disinvestment take account of these factors.

Political, Clinical, and Social Challenges to Removing an Established Technology (including challenges to limiting coverage to specific patients, institutions, or providers)

For existing technologies or practices there are complexities that do not beset those that are new or emerging. These relate to their entrenched status (in medicine and society) and include for example:

  • Resistance to change due to established clinical training and practice paradigms
  • Clinical and consumer influence and preferences
  • Political sensitivities, interests, and resistance
  • Supplier-induced demand
  • Incentive and disincentive mechanisms
  • The sunk costs of human and physical capital that would thereby become obsolete.

Schon23 describes how social systems work hard to resist change, a phenomenon he labels “dynamic conservatism.” Any inquiry must, therefore, factor in the many social and ethical issues otherwise ignored by pure technical rationality. Building on this, Lehoux24 advocates: “Research on health technology should therefore seek to make explicit the ethical, social, and political values embedded in a given innovation and reinforced through its use” (p. 171). One element of this has been illustrated by Elshaug et al.2 in reference to proposed changes to in vitro fertilization funding in Australia where a disinvestment about-face occurred as a result of pressure aimed at politicians from various interest groups. The example demonstrates that once a technology has diffused and is entrenched as an offering, public opinion and established interests can constrain the extent to which a regulatory authority can contain it or reduce access. Disinvestment has considerable potential to be a politically controversial agenda. For policy-makers to commit, there must be clear political support and strategies in place to handle anticipated backlash and/or lobbying. It is not a cost-free activity — the science requires financial and organizational commitment, and its success is dependent on politicians willing to lead from the front.

Lack of Published Studies that Clearly Demonstrate that Existing Technologies and Practices Provide Little or No Benefit (including to patient subgroups)

Complexities exist around adequate evidence in disinvestment review decisions. Given that in current HTA structures, the burden of proof is reversed so that the payer must demonstrate inferiority beyond reasonable doubt, there may be insufficient evidence to present a definitive case for removing or restricting an item. Disinvestment largely deals with entrenched, legacy technologies that were listed on reimbursement schedules prior to modern HTA processes. Once listed, there are few groups motivated, resourced, or directed to collect comprehensive and rigorous data. Often no policy group has been assigned a stake in the collection or generation of such evidence; hereto, it has generally accrued through the actions of clinical or research groups. Over time, a persuasive picture may build that certain technologies or practices hold low clinical and/or cost-effectiveness or that indication creep is occurring. But with an absence of evidence, decision-makers may be reluctant to remove funding (often under pressure from clinical groups), restricting a service that a minority of recipients might genuinely benefit from. A policy stakeholder interviewed by Elshaug et al.21 stated this challenge as follows:

The problem which MSAC comes up against all the time and I imagine any health technology agency or even any evidence-based assessment, like Cochrane, is that there may not be evidence at all, or there may only be limited evidence and so you’re having to base the decision on lack of evidence rather than evidence of lack of effectiveness . . . are you going to stop something because there’s a lack of evidence?

To overcome this challenge requires agreed criteria for decision-making where uncertainty is present. This interview respondent went on to state:

You would accept different levels of evidence because [for some practices] you’re never going to get higher levels... which is why people say it’s [MSAC] inconsistent, but when you look at it, it’s actually dealing with a different set of criteria almost.

Back to top


Identifying and Prioritizing Technologies and Practices for Evaluation

The best way would be to get a few examples of services which are clearly expensive, either in themselves or because of the volume…that could be brought to the attention of ministers and senior bureaucrats, with some notion of the resources that would be freed up if those procedures were at least limited or replaced. There is plenty of that sort of stuff done already. It’s just a case of pulling it together in way that they understand…. academics might have an interest in using their skills to represent the data in different forms (S3)*
*Participant pseudonym(s) provided (e.g., S9 = Stakeholder 9).
From Elshaug et al.21

To ensure a maximally productive approach, any process for selecting health care practices with a view to evaluating them for displacement should follow a protocol with prespecified, transparent selection criteria. Funding could be allocated to support a centralized approach — based on “horizon scanning” techniques — that would facilitate the systematic and transparent identification of existing, potentially ineffective practices on which to prioritize candidates for assessment as to their safety, clinical effectiveness, and, where appropriate, cost-effectiveness. The process could be jointly funded by all relevant stakeholders but centrally administered, with an HTA group resourced to undertake identification and assessment and to liaise with clinicians, consumers, and funding stakeholders. Final HTA reports of chosen candidates could be disseminated to provincial regions for contextualization to the local environment. See Box 1 and Box 2 for a detailed set of criteria to be used for this identification and prioritization process.

Developing New Approaches to Assessment

The existing methods of HTA are highly applicable to disinvestment, and recent work has indicated that stakeholders believe that HTA agencies have a central role to play in “re-evaluating existing programs.”25 However, the evaluation of entrenched technologies may require novel approaches to assessment. Subsequent to the NICE disinvestment agenda being released, a formal UK Treasury report into UK health research and funding reinforced the challenges faced in this area26:

The delivery of robust scientific appraisal for technologies is coming under increasing challenge as a result of its reliance on methodologies that, it is widely recognized, need further development, given that Health Technology Assessment (HTA) is a relatively new science. Appropriate research is required to address these challenges. In particular, research into methodology for…disinvestment methods (p. 103)

Berwick has suggested that it is time to “embrace a wider range of scientific methodologies”27 if the quality of medical care is to continue to improve. This may involve considering evidence beyond commonly accepted “gold standards,” invoking new sources of data, such as registries, and considering other pragmatic approaches to evidence collection and analysis. However, these approaches should be carefully considered and must not be at the expense of methodological rigour. To succeed in this field, HTA agencies will need to show commitment to quality and evidence-based best practice in their work.28 Similarly, coverage deliberations will need to explicitly consider all salient social, ethical, political, legal, economic, and clinical consequences of limiting or removing a service. For example, restricting services may influence human resource requirements and result in the need for staff retraining. This should not detract from pursuing the agenda but is an important consideration for the assessment process. In the absence of such consideration, disinvestment initiatives may underestimate the complexity of the task and the potential for both stakeholder opposition and diminished political support.

Box 1: Criteria for Identifying Existing, Potentially Non-cost-effective Practices as Candidates for Assessment*7

New evidence: New evidence on safety, effectiveness, and/or cost-effectiveness may come to light that changes previously held conclusions and is sufficiently useful for decision making. Sources include subsequent trials, cumulative meta-analyses, post-market surveillance, audits and registry data. It could also include longer-term datasets where evidence becomes available on patient-relevant outcomes, rather than surrogate outcomes used previously; and developments in diagnostic parameters (and treatment outcome measures) that have undergone evidence-based reclassification.

Geographic variations in care: Geographic variations (eg, the Dartmouth Atlas of Health Care), after adjusting for demographics and location of centres of excellence, suggest differences in clinical opinion about the value of the interventions.

Provider variations in care: Clinical heterogeneity of procedure, where the choice of intervention varies (eg, surgical variation) for the same class of disease or condition (seek coupling with evidence of a long learning curve, and inconsistent or operator-dependent safety and effectiveness).

Temporal variations in volume: A trend in item volume between time-points (eg, 2, 3 or 5 years), of a substantial percentage (say 30%, 50% or 80%). Most often this is a decrease. An increase may flag “leakage” or indication “creep” (usage beyond the restriction/indication-adjusting for trends in incidence).

Technology development: When an intervention has evolved to the point that it differs markedly from the initial or prototype intervention that was originally assessed or funded, then the initial intervention should be reviewed (eg, 256-slice compared with four-slice computed tomography). Note: this may be identified as a volume variation if marketing data are used, but not if the data source is a Medicare item number (Medicare describes the service, not the technical indications for undertaking that service). Perhaps an indicator that the unit cost of the intervention may be increasing unduly.

Public interest or controversy: Expressions (to media, letters to editors, inquiry submissions) from patients, consumer advocacy and support groups, and community groups, highlighting negative (or ineffective) experiences following treatment. To be substantiated by evidence.

Consultation: Consultation with clinical, nursing, allied health and technical staff, health care administrators and funders (including both public and private health insurance).

Nomination: A process (potentially anonymous) established where individuals, associations and colleges (from medical, nursing, research, allied health or the general public) could nominate interventions and justify their choice. To be substantiated by evidence.

Assess new intervention — displace old: When a new intervention is presented to the relevant committee(s) for regulatory assessment, and is considered a potential replacement for an established comparator(s) for that indication, then that comparator for that patient indication is automatically considered and assessed for disinvestment.

Leakage: Technology use (with reimbursement) outside the evidence-based indications (see also Temporal variation, above).

Legacy items: Long-established technologies that have never had their cost-effectiveness assessed — look for coupling with other identification items.

Conflict with guidelines: Where practice is inconsistent with clinical practice guidelines, clinical college position statements, Cochrane Review recommendations (and where there is no Cochrane Review on that technology).

Precedent: Where another authority has acted to review the appropriateness of a given technology

† The Medical Services Advisory Committee ( advises the Minister for Health and Ageing on evidence for safety and (cost)-effectiveness of medical technologies and procedures. This advice informs decisions about public funding. The Pharmaceuticals Benefits Advisory Committee (
/main/publishing.nsf/Content/ health-pbs-general-listing-committee3.htm#pbac
) assesses applications for listing of medicines on the Pharmaceutical Benefits Scheme.

Note: Box 1 is reproduced from: Elshaug AG et al. Identifying existing health care services that do not provide value for money. MJA 2009; 190 (5): 269-273. ©Copyright 2009. The Medical Journal of Australia — reproduced with permission.

Box 2: Criteria to Inform the Prioritization of Candidates for Detailed Review After Identification*7

Cost of service: High cost per procedure (eg, high item cost on the Medicare Benefits Schedule or Pharmaceutical Benefits Scheme), high cost by volume, or an aggregate measure of these.

Potential impact:

  • Likely health impact (eg, crude estimate of quality-adjusted life-years lost per patient).
  • Likely cost effects (eg, crude estimate of cost saving per patient; liberation of additional resources, including downstream costs such as theatre time required for corrective procedures, and sunk costs of human and physical capital, including costs of retraining, and costs associated with length of hospital stay).
  • Overall assessment relating to the maintenance of equity in care should this health care intervention be displaced (eg, access by patient subgroups).

Cost-effective alternative: When a cheaper but more, or equally, effective alternative exists, is identified or emerges. See also Box 1 item Assess new intervention — displace old.

Disease burden: Conditions associated with low degrees of disability or morbidity or low rates of mortality (but excluding orphan conditions) may influence priority differentially to those with high degrees or rates. “Low” may reduce the potential for controversy; “high” may represent greater scope for reinvestment/reallocation of resources.

Sufficient evidence available: Rigorous assessment requires robust evidence on which decisions can be made. While evidence is rarely 100% conclusive, it should be available and adequate to offer decision-making utility.

Scope for time-limited funding with “pay for evidence” or “only in research” provisions: If there is not new, adequate or sufficient evidence, but other criteria are met and/or there is a moderate indication of (cost)-ineffectiveness within existing evidence, then there should be scope for ‘”(time-limited) funding with evidence generation” to assist decision making. Time-limited funding (standard subsidy) is conditional on patients being enrolled in a controlled clinical trial. Internationally, this is known as “pay for evidence”, “only in research” or “coverage with evidence development”. The need or extent of new evidence required to meet “sufficient evidence” (item above) might inform prioritisation.

Futility: An intervention that is highly unlikely to result in “meaningful survival” or benefit. For example, life-saving treatments for the seriously demented (especially those who have given advance directives) or procedures that require multiple stages to which patients have poor adherence due to pain or side-effects; and treatments with high relapse rates.

*It is likely that points of identification would be incorporated in parallel into assessments of prioritisation, as determined by decision makers.

Note: Box 2 is reproduced from: Elshaug AG et al. Identifying existing health care services that do not provide value for money. MJA 2009; 190 (5): 269-273. ©Copyright 2009. The Medical Journal of Australia — reproduced with permission.

Clinical Governance Approach: Guidelines

[Guidelines] are good in theory but in practice, unless there’s a political reason for wanting to have that therapy assessed it doesn’t get done. (S4)*
*Participant pseudonym(s) provided (e.g., S9 = Stakeholder 9).
From Elshaug et al.21

Clinical practice modification and refinement, perhaps through clinical practice guideline development and implementation, have demonstrated efficacy. However, in the area of disinvestment, NICE has experienced that this approach resulted in limited and/or differential adoption — demonstrating that it is generally inadequate in this context (see) Box 3.

Box 3: Failure of Guidelines to Limit the Overprescribing of Proton Pump Inhibitors for Dyspepsia

Studies consistently show that proton pump inhibitors (on which 7 billion British pounds was spent globally in 2006) are being overprescribed worldwide in primary and secondary care. Between 25% and 70% of patients taking these drugs have no appropriate indication. Proton pump inhibitors cost more than other agents, yet effective and less expensive alternative drugs, such as H2-receptor antagonists, are available.29 Batuwitage30 conducted a prospective analysis of the impact of NICE (and additional) guidance that was introduced in an attempt to reverse this trend. They aimed to measure any reductions in inappropriate use from pre- versus post-guidance implementation. The pre-intervention analysis revealed that 54% of patients were receiving proton pump inhibitors inappropriately. Six months after the intervention (guidelines), 51% of patients were receiving proton pump inhibitors outside the appropriate indications. The intervention used in this study had no effect on the proportion of patients taking a proton pump inhibitor at the time of hospital admission or on the appropriateness of prescribing in the community.

Policy Approach: Reimbursement Levers

I actually think for things that are dramatically wrong, reimbursement structures are probably the way in which you’re going to make the biggest improvement happen, but there’s no good track record in this country [Australia] of that. (S7)*
*Participant pseudonym(s) provided (e.g., S9 = Stakeholder 9).From Elshaug et al.21

Removal from funding schedules: For some technologies or practices, the evidence for safety and/or effectiveness is convincingly negative, yet the practice persists. In these instances, complete removal (from funding schedules) may be appropriate. Insurers, primarily in the USA, have set precedents in this area.

Tighten or restrict indications associated with reimbursement: Often evidence mounts about the population subgroups who achieve the most, and least, benefit from certain technologies or practices. Reimbursement indications can be tightened to target those with the greatest capacity to benefit. Auditing processes can be used to identify and limit indication creep.

Restrict providers to “centres of excellence”: High volume specialist providers (for example) may achieve superior outcomes. In these cases, reimbursement may be restricted to centres of excellence. Equity of access is a consideration here.

Partial reimbursement: For practices known to deliver less than optimal outcomes (but a degree of benefit), only partial reimbursement may be provided to represent the degree of health outcome achieved. Rule of rescue should be considered.

Risk-sharing (practitioner reimburses payer): Internationally, agencies are considering or have adopted policies of standard, full reimbursement and tracking of individual health outcomes following procedures or treatment. If the outcomes are below that expected, then the health care practitioner reimburses the payer a previously agreed-upon amount.

Reimbursement only for guideline adherence: Internationally, agencies are now considering reimbursing only if practice adheres to appropriately endorsed clinical practice guidelines. Any practice that deviates is reimbursed only once judged before a panel of peers as “acceptable deviation.” Such an approach relies on electronic systems for documentation of clinical activities and may be feasible for only a limited number of technologies.

Compulsory review of all technology, however introduced: On occasion, technologies that have not been government-purchased or commissioned enter the system (for example by charities, left over from research, or donated by manufacturers). Whereas there may be limited initial outlay, they require training resources and ongoing maintenance (consumables) and consume staff time. Importantly, this process may circumvent standard HTA avenues, thus having implications for future disinvestment. All technologies to be introduced (by any avenue) should, therefore, be subject to standard HTA review and decision processes.

Sunset clauses: Where a health care practice or technology has questionable clinical or cost-effectiveness, but insufficient evidence exists for decision-making (i.e., there is substantial uncertainty) then funding should be (a) guaranteed for only a set time period into the future and, where appropriate, (b) conditional on evidence generation (see Box 4.)

Concurrent specification of what is to be removed from funding when any new practice or technology is first funded: Gafni and Birch31 have called for direct comparisons to be made “between the incremental benefits associated with the new program and the incremental benefits associated with those programs that must be cancelled or reduced in order to generate the additional resources required by the new program.”

Box 4: Coverage with Evidence Development

Substantial challenges exist, particularly around adequate and timely definition and acceptable proof of inferiority. This is not only conceptually difficult but is also limited by data availability and interpretation. Further complicating this is the lag that often exists in the reliable reporting of health outcomes in routine clinical practice. The main problem for decision-makers is how to balance the costs of waiting for better evidence against the costs of acting prematurely (both in potentially wasted health gain and wasted resources).

In such cases, there may be potential for coverage with evidence development (CED) or funding with evidence generation mechanism (called “only in research” [OIR] in the UK). This approach is currently applied for some emerging technologies but could be adapted for existing practices. Here, payers and/or regulators may allow ongoing funding for only a defined period of time (and perhaps to limited providers) to allow for the generation and/or analyses of “necessary” evidence.

An important point here is that this mechanism reverses the burden of proof, placing it back on to the sponsor (not the payer), as occurs in pharmaceuticals. Payers have been pressured to continue funding simply because of absence of evidence rather than clear evidence of absence (of benefit). Legacy items on reimbursement schedules may have got onto the system before rigorous methods of review were applied. If today’s rigour had been applied, some may not have been approved. Despite being reimbursed for many years — the evidence generated over time may not withstand scrutiny. Ethically, this is no reason to simply quarantine or embargo such items from investigation.

This debate is developing rapidly and Chalkidou et al.32 provides salient points:

  • “For OIR recommendations to be a realistic policy option, the gap between coverage decisions and clinical decisions must be closed” (p. 1650)
  • “None of the more recent CED decisions has yet produced sufficient data to support reconsideration of coverage” (p. 1650)
  • “Nor should the OIR option be used as an excuse to disseminate clinically ineffective or cost-ineffective innovations under pressures from stakeholders to prevent a negative coverage decision” (p. 1651).

Back to top


Note: The following are elements of a proposed approach to the management of obsolescence in health care and are provided in the spirit of fostering debate.

Element 1: High-level decision and commitment is required to make disinvestment an explicit, formal, and resourced policy agenda. This would involve the development of partnerships involving government (provincial and federal), professional colleges, and relevant stakeholder groups including patient and citizen groups to further a disinvestment agenda by fostering awareness raising, collaboration, and improved health outcome data generation and reporting.

Element 2: A regulatory framework for disinvestment decision-making is required that is transparent and removed from vested interests. This may be parallel to existing processes for new and emerging technologies. Elements of the framework may include explicit consideration of formally decommissioning old technologies and practices as new items are approved if there is adequate evidence.

Element 3: Given the strain that exists on committees responsible for new and emerging technologies and practices, consider either (a) additional resources and capacity for those committees to consider existing items in parallel or (b) the establishment of new, parallel committee(s) to consider existing items.

Element 4: Regulatory support should be provided for HTA recommendations for (a) removing or (b) reducing reimbursement or (c) restricting use of a comparator technology if a new or existing item has better clinical or cost-effectiveness for a given indication. These analyses and decisions are dependent upon the maintenance of equity in care.

Element 5: To ensure a maximally productive approach, any process for selecting health care practices with a view to evaluating them for displacement should follow a protocol with prespecified, transparent selection criteria. Funding could be allocated to support a centralized “horizon scanning” style approach that would facilitate the systematic and transparent identification of existing, potentially ineffective practices on which to prioritize candidates for assessment as to their safety, clinical effectiveness, and, where appropriate, cost-effectiveness. The process could be jointly funded by all relevant stakeholders but centrally administered, with an HTA group resourced to undertake identification and assessment and to liaise with clinicians, consumers, and funding stakeholders. Final HTA reports of chosen candidates could be disseminated to provincial regions for contextualization to the local environment.

Element 6: Sections 4.3 and 4.4 indicate there is panoply of options in terms of guideline and/or reimbursement levers to effect disinvestment. Debate is essential among all relevant Canadian decision-making stakeholders as to which of these mechanisms, or combinations thereof, are most appropriate within a given jurisdiction at impacting effective disinvestment. However, international experience of the impact of HTA processes strongly supports the need for leverage at the reimbursement level to effect positive and lasting reform.33,34

Element 7: A dedicated stream of funding for capacity building in research and policy development in disinvestment is required. An explicit disinvestment agenda will require the development of new and transparent methods to dovetail with existing HTA capacity. This will require initial capital input to support stakeholder consultations, a working disinvestment development and implementation plan, and policy reform. Pilot funding should be provided to a Canadian organization with appropriate skills, knowledge, and broad-based oversight to commence this work.

Back to top

CONCLUSIONS that you can, within the existing envelope, offer at least a no more expensive and arguably cheaper structure and tag with it better quality…line the dollars up with the quality, then I think they’ll be interested…(S10)*
Notes: *Participant pseudonym(s) provided (e.g., S9 =Stakeholder 9).
From Elshaug et al.21

Priority setting in health care is complex. However, clinical, economic, social, and ethical evaluations of potentially ineffective or redundant health care practices will generate evidence that can be used to make explicit the assumptions and consequences of (political) decision-making. As stated by Campbell, “attempts to draw attention to the complexity of the task contribute to increased accountability for the process35 (p. 381). The intended beneficiaries of an expanded agenda in this area include all consumers, purchasers, and providers of health care services through a systematic policy approach to improved quality and safety of care and economic savings that can be redistributed to areas of greater gain within the health care system.

We can sometimes improve the health system by making it more efficient and accountable while enhancing quality of care, without necessarily using additional resources. A recent report into disinvestment planning by NICE showed that there are risks associated with identifying practices and technologies as ineffective in some circumstances, as there may have been considerable training and infrastructure development and some stakeholders may prefer that the status quo be maintained.36 The report concludes that “these are not reasons not to ask the difficult questions nor to shy away from making unpopular recommendations.” Given these sensitivities, approaches such as those proposed here may improve consultation, transparency, and overall governance. Canada has an international reputation for its capacity and governance in HTA processes. It is well placed to tackle these challenges head-on and forward this important agenda. The answers have the potential to enhance the sustainability and quality of Canadian health care.

Back to top

APPENDIX 1: Literature Search Methods

A peer-reviewed literature search was conducted by an information specialist with input from the project team.

Published literature was identified by searching the following bibliographic databases: BIOSIS Previews, EMBASE and MEDLINE through Ovid, and PubMed.

The search strategy consisted of both controlled vocabulary, such as the National Library of Medicine’s MeSH (Medical Subject Headings), and keywords. The main search concepts were obsolescence, policy, and health technologies. A supplementary search was run without the health technologies concept.

The search was restricted to English language articles published from 1999 onwards. The last search was run on June 12, 2009.

Grey literature (literature that is not commercially published) was identified by searching the websites of HTA and related agencies, professional associations, and other specialized databases. Google and other Internet search engines were used to search for additional web-based materials and information.

These searches were supplemented by hand-searching the bibliographies of key papers as well as related reference searching in PubMed. Additionally, articles that cited the key papers were identified through citation searching in Web of Science.

Back to top

APPENDIX 2: De-insurance Tables from Canadian Provinces and Territories, 1990s

Table 1: De-insured Health Care Services by Province and Territory (from CMA, 2001)1 (does not include services that have been restricted by provider or location of service provision)

Service* Province and/or Territory
2nd or subsequent ultrasound in uncomplicated pregnancy NS
All services associated with de-insured services NB
Artificial or intrauterine insemination NS
Audiology services Ont.
Breast reduction or augmentation NS, NB
Chiropractic services Sask.
Circumcision (newborn) / circumcision (unspecified) Nfld., NS, NB, Alta., YT / PEI, Ont.
Correction of inverted nipple NB
Cosmetic surgery Alta.
Dental anesthesia for uninsured dental procedures Sask., Alta.
Destruction of hair follicles Ont., Sask.
Diagnosis of bone fracture without reduction NS
Diagnostic punch or excision biopsy of skin and mucous membrane NS
Ear wax removal NS
Epilation of facial hair PEI
Excision of benign, superficial cysts, lipomas, pimples etc. NS, Ont., BC
Excision of xanthelasma Nfld., NS
Eye examinations PEI, Sask. (people aged 18+ years),
Alta (people aged 19 to 64 years)
Eye refractions Nfld., Sask.
Gastroplasty or gastric bypass NS, NB
Hypnotherapy Nfld.
In vitro fertilization Ont.
Intrauterine insemination NB
Laser ablation Sask.
Lipectomy and apronectomy NS
Non-surgical treatment of temporomandibular joint disorder Alta.
Otoplasty PEI, NB, Alta.
Penile prosthesis Sask.
Physiatric supervision NS, Que. (partial)
Removal of skin lesions NB, Sask.
Removal of tattoos Ont., Sask.
Removal of warts NS; Alta.; BC
Repair of earlobes torn by earrings Ont.
Reversal of sterilization PEI, NB, Ont., Sask., Alta.
Routine vision care for people aged 10 to 19 years NS, NB
Sclerotherapy Que., Ont.
Sex re-assignment surgery Ont.
Skin scraping for fungi with potassium hydroxide NS
Subsequent injections for impotence NB
Surgical assist for cataract surgery and lens insertion NS
Tongue tie NS
Transcranial Doppler BC
Travel immunizations Ont.
Vasectomy reversal NB, Sask.
Venipuncture NB, Ont.
Weight-loss counselling and therapy Ont.

*Some exceptions may apply. Please refer to the complete CMA reference.1
No information is available on de-insured services in Manitoba or the Northwest Territories.
“Catalogue of uninsured medical and medically-related services” — Adapted from Canadian Medical Association June 2001 by permission of the publisher. © 2001 Canadian Medical Association.

Table 2: De-insured Physician Services, Ontario Health Insurance Plan, 199413

Recommended for de-insuring:
  • In vitro fertilization (for other than total fallopian tube blockage)
  • Injection of simple varicose veins
  • Removal of acne pimples
  • Removal of certain benign lesions
  • Removal of tattoos (except resulting from abuse)
  • Repair of torn earlobes (except from acute trauma)
  • Reversal of sterilizations
  • Routine newborn, ritual, or cosmetic circumcision
Recommended not de-insuring:
  • Annual health examinations
  • Excision of calcaneal spurs
  • General anaesthetic for uninsured dental procedures
  • Insertion of penile prosthesis for impotence
  • Insertion of testicular prosthesis
  • Intracorporeal injection for impotence
  • Otoplasty to correct outstanding ears
  • Removal of port wine stains on the face and neck
  • Travel assessments/immunization clinics
  • Uvulopalatopharyngoplasty
No recommendations:
  • Weight loss clinics
Table reproduced from: MK Giacomini, "The Which-Hunt" in Journal of Health Politics, Policy and Law, Volume 24, no. 4, p. 730. Copyright, 1999, Duke University Press. All rights reserved. Used by permission of the publisher.

Back to top

APPENDIX 3: About the Authors

Dr. Adam Elshaug (BA, BSc[Hons], MPH, PhD) holds a Hanson Institute Research Fellowship with Adelaide Health Technology Assessment (AHTA) and is Senior Lecturer in the Discipline of Public Health, School of Population Health and Clinical Practice at the University of Adelaide, Australia. He trained in clinical epidemiology and health services and policy research. His PhD (awarded December 2007) and postdoctoral research has focused in the area of disinvestment — clinical and policy barriers to disinvestment and policy reform and implementation. His PhD received a special commendation and generated six peer-reviewed publications and seven awards as an emerging researcher. It also led to a successful project grant application (The ASTUTE Health Study; 2009-2011) valued at almost $900,000 (NHMRC) on which Dr. Elshaug is a chief investigator and project manager.

Dr. Elshaug is gaining significant international recognition for his contribution to this area of research and policy development. He has received over 40 invitations to address conferences as well as government, academic, insurance, and health technology assessment groups in Australia and internationally. Dr. Elshaug is working with numerous bodies in Australia (federal and state), Spain, the UK, Canada, and the USA to advance policy reform, and he is collaborating extensively on research projects.

Ms. Amber Watt (BMedSci, GDPH) holds a Bachelor of Medical Science and a Graduate Diploma of Public Health. She has a professional background in health technology assessment and systematic review, having worked as a research officer for the Royal Australasian College of Surgeons for three and a half years. Ms. Watt is now a senior research officer within the ASTUTE Health Study, an Australian-based research project that aims to design, implement, and evaluate a model to identify the social, ethical, political, economic, and epidemiological factors that perpetuate the use of ineffective health care practices and to test if practices can be disinvested. She is the author of a number of peer-reviewed journal articles and official reports for the Australian Government Department of Health and Ageing.

Assoc. Prof. John Moss (MSocSci, BEc, MBBS, FCHSE, FPHAA) is the Head of the Discipline of Public Health in the School of Population Health and Clinical Practice at the University of Adelaide. He has established an extensive research program in health economics and in health services (delivery) research. The emphasis is on prevention, primary health care, and the hospital-community interface. He has been joint chief investigator of eight successful category 1 competitive grant applications (NHMRC/ARC); and has been author or co-author of 53 peer-reviewed journal publications.

Assoc. Prof. Moss is a joint team leader in providing consultancy services to the Australian Government Department of Health and Ageing to evaluate major submissions by pharmaceutical manufacturers to the Pharmaceutical Benefits Advisory Committee. To date, 64 commentaries have been completed. He has also overseen the health economic aspects of nine consultancies for the Medical Services Advisory Committee (MSAC). At the University of Adelaide, Assoc. Prof. Moss teaches courses in Public Health Theory and Practice and in Health Economics and modules on the structure and function of the Australian health system and on health inequalities. He has supervised to completion six doctorates and 22 masters and honours theses and dissertations.

Prof. Janet Hiller (BA, DipSocStudies, MPH, PhD, FPHAA) holds the Chair of Public Health at the University of Adelaide and is Deputy Head of the School of Population Health and Clinical Practice. She is the founder and Director of Adelaide Health Technology Assessment (AHTA), the leading health technology assessment group in the southern hemisphere and the only agency in Australia that conducts HTA and associated research along the continuum from early identification to disinvestment. She has a PhD in Epidemiology (John Hopkins) and an MPH (Hebrew University).

Prof. Hiller has more than 110 publications with a focus on the evaluation of health services associated with pregnancy and childbirth and, in more recent years, on HTA, but also including cancer screening, environmental and infectious epidemiology, and health inequalities. She has been a chief investigator on numerous competitive grant applications and has received financial support for her research from state, national, and international organizations. She is an active contributor to research policy development in Australia, serving on a number of research committees. In addition, Prof. Hiller teaches courses on Public Health, Epidemiological Research Methods, Health Technology Assessment, and Screening. She has supervised more than 60 students to completion, including 13 doctorates, two post-doctorates, and 50 masters and honours theses and dissertations.

Back to top


  1. Catalogue of uninsured medical and medically-related services. Ottawa: Canadian Medical Association; 2001 Jun.
  2. Elshaug AG, Hiller JE, Tunis SR, Moss JR. Challenges in Australian policy processes for disinvestment from existing, ineffective health care practices. Australia and New Zealand Health Policy 2007;4(1).
  3. Joshi MP, Stahnisch FW, Noseworthy TW. Reassessment of health technologies: obsolescence and waste [unpublished internal manuscript]. Ottawa: Canadian Agency for Drugs and Technology in Health; 2008.
  4. Rice TH. The impact of changing medicare reimbursement rates on physician-induced demand. Med Care 1983;21(8):803-15.
  5. White KL. Evidence-based medicine. Lancet 1995;346(8978):837-8.
  6. Smith R. Where is the wisdom...? BMJ 1991;303(6806):798-9.
  7. Elshaug AG, Moss JR, Littlejohns P, Karnon J, Merlin TL, Hiller JE. Identifying existing health care services that do not provide value for money. Med J Aust 2009;190(5):269-73.
  8. Strech D, Persad G, Marckmann G, Danis M. Are physicians willing to ration health care? Conflicting findings in a systematic review of survey research. Health Policy 2009;90(2-3):113-24.
  9. Culyer AJ. The morality of efficiency in health care--some uncomfortable implications. Health Econ 1992;1(1):7-18.
  10. Mooney G. Key issues in health economics. London: Harvester Wheatsheaf; 1994.
  11. Daniels S, Ramey M. The leader's guide to hospital case management. Sudbury (MA): Jones and Bartlett Publishers; 2005.
  12. Schoen C, Osborn R, Doty MM, Bishop M, Peugh J, Murukutla N. Toward higher-performance health systems: adults' health care experiences in seven countries, 2007. Health Aff (Millwood) 2007;26(6):w717-w734.
  13. Giacomini MK. The which-hunt: assembling health technologies for assessment and rationing. J Health Polit Policy Law 1999;24(4):715-58.
  14. Mitton C, Donaldson C. Resource allocation in health care: health economics and beyond. Health Care Anal 2003;11(3):245-57.
  15. Mitton C, Patten S, Donaldson C, Waldner H. Priority-setting in health authorities: moving beyond the barriers. The Calgary experience. Healthc Q 2005;8(3):49-55.
  16. Cotter D. The National Center for Health Care Technology: lessons learned. In: Health Affairs blog. Bethesda (MD): Project HOPE; 2009. Available: (accessed 2009 Jul 8).
  17. Banta HD, Thacker SB. The case for reassessment of health care technology. Once is not enough. JAMA 1990;264(2):235-40.
  18. Neumann PJ, Kamae MS, Palmer JA. Medicare's national coverage decisions for technologies, 1999-2007. Health Aff (Millwood) 2008;27(6):1620-31.
  19. House of Commons Health Committee. National Institute for Health and Clinical Excellence: first report of session 2007–08. Volume 1. London: The Stationery Office; 2008 Jan 10.
  20. Australian Government. Budget paper no. 2, 2009-10. Part 2: Expense measures - health and ageing. Barton: Commonwealth of Australia; 2009 May 12. Available: (accessed 2009 Jul 8).
  21. Elshaug AG, Hiller JE, Moss JR. Exploring policy-makers' perspectives on disinvestment from ineffective healthcare practices. Int J Technol Assess Health Care 2008;24(1):1-9.
  22. Wilensky GR. Developing a center for comparative effectiveness information. Health Aff (Millwood) 2006;25(6):w572-w585.
  23. Schon DA. Beyond the stable state: public and private learning in a changing society. London: Temple Smith; 1971.
  24. Lehoux P. The problem of health technology: policy implications for modern health care systems. New York: Routledge, Taylor & Francis Group; 2006.
  25. Hivon M, Lehoux P, Denis JL, Tailliez S. Use of health technology assessment in decision making: coresponsibility of users and producers? Int J Technol Assess Health Care 2005;21(2):268-75.
  26. Cooksey D. A review of UK health research funding. London: Stationery Office; 2006.
  27. Berwick DM. The science of improvement. JAMA 2008;299(10):1182-4.
  28. Chalkidou K, Tunis S, Lopert R, Rochaix L, Sawicki PT, Nasser M, et al. Comparative effectiveness research and evidence-based health policy: experience from four countries. Milbank Q 2009;87(2):339-67.
  29. Forgacs I, Loganayagam A. Overprescribing proton pump inhibitors. BMJ 2008;336(7634):2-3.
  30. Batuwitage BT, Kingham JG, Morgan NE, Bartlett RL. Inappropriate prescribing of proton pump inhibitors in primary care. Postgrad Med J 2007;83(975):66-8.
  31. Gafni A, Birch S. Incremental cost-effectiveness ratios (ICERs): the silence of the lambda. Soc Sci Med 2006;62(9):2091-100.
  32. Chalkidou K, Lord J, Fischer A, Littlejohns P. Evidence-based decision making: when should we wait for more information? Health Aff (Millwood) 2008;27(6):1642-53.
  33. Morgan SG, McMahon M, Mitton C, Roughead E, Kirk R, Kanavos P, et al. Centralized drug review processes in Australia, Canada, New Zealand, and the United kingdom. Health Aff (Millwood) 2006;25(2):337-47.
  34. O'Malley SP. The Australian experiment: the use of evidence based medicine for the reimbursement of surgical and diagnostic procedures (1998-2004). Aust New Zealand Health Policy 2006;3:3.
  35. Campbell D. Prioritizing health-care funding: all things considered. Intern Med J 2005;35(7):379-81.
  36. Kelly M. Public health programmes and interventions and NHS disinvestment. London: Centre for Public Health Excellence. National Institute for Health and Clinical Excellence (NICE); 2006. Available: (accessed 2009 Jun 25).


Back to top