NoTeb : NORDIC exploration of an early assessment framework in health care

Nordic Test Beds (NoTeB) is a Nordic Innovation funded project to strengthen Nordic health care cooperation. By joining Nordic university hospitals and innovation centers, its aim has been to provide a Nordic test bed cooperation ecosystem that includes templates and tools for early decisional support. The objective of this study was to describe how the partners in the Nordic collaboration developed and agreed upon an early assessment framework in health care. The framework was also presented as a user guide with the aim of providing advisory guidelines accessible to health care institutions to support early assessment of health innovation. A co-creation process comprising all collaborating Nordic partners initiated by a workshop, sharing current practice and aligning needs and content of for a decision support tool. Large Nordic variation in value assessment approaches were found. For the decision-making tools, two important features were emphasized; the need for a decision support rather than a decision-making tool and that the tool should be based on valid measures; HTAmethodology. For the user guide, it was emphasized that it should be easily available in an easy-to-read report format. In conclusion, NoTeB was successful in uniting Nordic countries in a common objective to develop a decisionmaking tool and a user guide to assess health innovations. Although the tools and the guide are still to be tested, this is a first step in developing a standardized tool for innovation assessment among the Nordic countries.


Introduction
The global health care industry faces two critical challenges: a large variation in patient outcomes and continually increasing costs.Health care constitutes a significant part of public sector expenditures.Total government expenditures among OECD countries was 41% of GDP in 2015, and health typically accounts for around 20% of these expenditures [1].These numbers and trends make innovation imperative if health care is to solve societal problems efficiently [1,2].
There is lack of diffusion and adoption of innovation and a need for value-based health care delivery [3,4].Value-based care delivery focus on patient health outcomes and the costs of delivering these, instead of feefor-service payment.It stimulates developers of health and welfare solutions and the health care industry as 11.3.2019 FinJeHeW 2019;11(1-2) 101 such, as cooperation between health care providers and health care industry on new service models, is essential [5].
The Nordic health care systems vary in their service production and operational procedures from country to country.Also, testbeds; the testing environments where companies and health care collaborate, operate with different methods and practices.Moreover, the companies and developers of health and welfare solutions within the Nordic region have indicated that they do not have a sufficient understanding of the needs of the public sector healthcare actors.There is a need to strengthen the Nordic level of strategic and operational cooperation in the health care field to help managerial decision making [6].The Nordic Test Beds project (NoTeb) was set up to address these challenges.By joining experiences and know-how from five Nordic university hospitals its aim has been to provide a Nordic test bed cooperation ecosystem that includes early decisional support.The present study addresses how early decisional support tools were created.Based on former experience with value assessments of health innovations [7], Oslo University Hospital, the Norwegian NoTeb partner, was provided responsibility for the early assessment framework.
The objective of this study was to describe the how the partners in the Nordic collaboration developed and agreed upon an early assessment framework in health care.The framework was also presented as a user guide with the aim of providing advisory guidelines accessible to hospitals and health care institutions with templates and tools to support early cost and benefit assessments for managerial decision support.

Setting: Nordic Test Beds (NoTeB)
The NoTeB project was set up to create a health product and service testing collaboration between companies, research institutes and health care professionals in Nordic health living labs connected to university hospitals [8].Employees at the Oslo University Hospital are responsible for more than half of all medical publications in Norway.However, in 2007, while publishing 350 scientific papers, only one innovation idea was reported.Therefore, the Innovation clinic was launched in 2007 to foster innovation by soliciting ideas from health care professionals, research companies, and others outside of the health care industry [9].One of the strategic goals of the Innovation clinic has been to develop methodology that succeeds to communicate and document the benefits of innovations.A series of 11 innovation projects gradually strengthened its ability to provide decisional support on how a new solution may benefit employees, patients, their families, the hospital, as well as society/the health care sector [7].This is crucial, both to convince decision makers related to single projects, to create attention for innovation across the organization, and to produce arguments for organizational and inter-organizational changes when needed.
The methods used are based on experience from 10 years of testing from Innovation clinic.In addition, Oslo University Hospital has collected testing procedures from all our partners to compare methods and approaches.Starting 2016, the NoTeb partnership was moved to the new consortium for innovation C3 -Centre for Connected Care, a Centre for research-based innovation [10].

FinJeHeW 2019;11(1-2) 102
A systematic literature search was carried out to identify methods for assessment of health innovation, with particular focus on early health technology assessment.
In total 1064 articles matched the search strategy and 39 articles matched the predefined inclusion criteria.
The included articles were reporting on early assessment of innovation in the health sector and articles reporting on methods or practice for early assessment of health innovations.

Stakeholder insight and expert opinion
Stakeholders from all collaborating Nordic partners, comprising hospitals, industry, users and primary care, were united to share experiences and needs in the development of an early assessment framework.Experts in the field of value assessment and innovation were present among the stakeholders.The stakeholder insight was initiated by a workshop, sharing current practice and aligning needs and content for the decision support tool.Common expectations expressed by the stakeholders were the need to share knowledge and experiences, obtain a mutual of the findings at Oslo University hospital and enable effective knowledge transfer between the partners.Before completion of the framework and the user guide a SWOT analysis was also carried out among the stakeholders.Promotion and sharing of innovation with the use of a "common language" and reduction of uncertainty were highlighted on the strength side.As for the threats, elements such as lack of resources and challenges due to different cultures were prominent.The stakeholders enhanced power of using the same methods and that early assessment enables earlier investments as opportunities.While the fact that the framework still holds much uncertainty and limited clinical evidence as weaknesses.

Context for implementation
The early assessment framework and the accompanying are intented to be implemented in all the participating organizations.The organizations are all users or devel-opers of health and welfare solutions within the Nordic region.

Pre-specified preliminary outcomes
Through former experience of value assessment of innovations at Oslo University Hospital and Nordic cases presented as part of the test bed cooperation system, an early support framework and a user guide was developed and reviewed by all Nordic partners prior to completion.The intention of the framework and the guide were threefold: 1) to provide proper early-stage innovation measures, 2) to provide a stage-wise template for how to initiate early health assessment and 3) to demonstrate throughout the user guide how earlystage measures may highlight innovation value and support decision-making.Valid measured based on Health Technology Assessment (HTA) was the methodology of choice.HTA is defined as an interdisciplinary process for synthesizing information regarding medical, social, economic and ethical issues related to the introduction of a new health technology [11].It is interdisciplinary and combines elements from 'cost-effect analysis' [12] with concepts such as 'benefits realization' and socio-economic analyses [11,13,14] and is the common assessment method within health care.In the literature of early health technology assessment stakeholder insight is used to assess potential benefit of health innovation [15][16][17][18].
The aim of the accompanying user guide was to provide advisory guidelines accessible to hospitals and health care institutions with templates and tools to support early cost and benefit assessments for managerial decision support.For educational purposes, the HTA methodology is simplified and applied for measurements also at later stages of the innovation process.Insight from a case study on wound treatment at Oslo University Hospital, "the Wound Support Network", was applied throughout the user guide both to demonstrate health assessment principles and to illustrate how implementation barriers of health innovations may be addressed.The latter innovation is described elsewhere in another context [19].

Results
The Nordic co-creation process Figure 1 shows a summary of input from the co-creation process with all participating Nordic partners.The discussion revealed large Nordic variation in value assessment approaches.For the decision-making tools, two important features were emphasized; the need for a decision support rather than a decision-making tool and that the tool should be based on valid measures; HTAmethodology.For the user guide, it was emphasized that it should be easily available in an easy-to-read report format preferably with case illustrations.The ability of the tool to share and discussed cases across the Nordic boundaries was expressed.
Early assessment framework and user guide was developed according to the recommendations above; the user guide is written in a report format, have case illus-trations and can be downloaded from the NoTeb website [20].

Highlights from the user guide: decision support tools and methodology
Based on HTA-methodology in local decision-making, benefit is what is best for the service, patients, employees, the hospital and society [11].These four domains constitute the assessment framework.Figure 2 presents the four domains of measurement, how they are categorized and presents examples of how data may be collected to inform decisions in each of domains.It also emphasizes that the measures in each case will vary by the project's indication and aim.The measures (Figure 3) include patient benefit, benefits to employees, financial effects locally for a hospital or health unit and nationally for the health sector, treatment effect/health effect, the risk associated with starting to use new things, organizational consequences in the form of improved flow within and between departments.The stakeholders participating in the design of the project define the content in each category.This will depend on the type of innovation and the objective.
Decision support methods will vary by innovation stage.Due to lack of data in early phases and according to the literature, stakeholder insight and scenario building play an important role in early innovation phases [15][16][17][18].
We recommend three questions to be answered ahead of the launch of an innovation project: 1. What is the most important bottleneck in the identified issue?Bottleneck or detailed process flowchart mapping can be used.

Which stakeholders are affected by a change,
and what approach should we use at this stage to obtain the information necessary to further develop the idea?Gather stakeholders in workshop, focus groups or interviews with experts.
3. On the basis of the problem we want to solve, what does already exists?It is important to enlist help in conducting a good and systematic literature search and map existing alternatives/products.
While benefits are independent of stage of innovation, the methodological approach varies by innovation stage.An overview of decision support methods is shown in Figure 4.It is highly recommended that stakeholder analysis and literature review are included when benefits are assessed at concept stage, that feasibility methodology; utility, acceptability and usability measures form the bases for pilot estimates and that before-and after studies and case control designs are reserved for the stages when the innovation is fully developed.

Discussion
The presented decision support tool and the content of the accompanying user guide is a result of a Nordic cocreation process and serves two purposes; a guide for sharing practice and a tool that allows early decision making in the introduction of innovative health care products and services, also outside their country of origin.
The user guide was developed to meet the need for tools to address and communicate health value and provide decision makers with information that help inform decisions.There is a clear lack of adoption, diffusion and knowledge transfer of innovations at local, regional, national and Nordic levels [3,21].Although innovation in health care and service innovation in particular is increasingly agreed upon, there is still uncertainty what strategic tools should support this ambition.Hierarchies and silos are found to be barriers and the capacity of public health care organisations to manage both exploitation and exploration has proved difficult [6].Also, innovation culture is about mindset for change and a managerial responsibility to provide a learning culture to succeed innovating [22].As there are great organizational similarities in the health care systems of the Nordic countries, ability to unite and standardize would be of great benefit.Current government systems, including healthcare, has not been built for flexible adaption to individual users' needs, which is now being called for [3].Accordingly, there is a need for developing a system of diffusion and knowledge sharing between health care institutions and facilitate test beds that includes the health care industry.For the latter to occur, information on how to assess value needs to be available at all hospital and health care institutional levels, including decision makers.We have a common responsibility to facilitate adoption of value-based health care.One such contribution to adapt to needs is the development of a user guide in an easy-to-read format with case illustrations.In the development of the user guide, insight from a former case-study on wound treatment at value-assessed at Oslo University Hospital was used to address how value assessment can be performed [23].In addition, the case was considered instructive and illustrative in demonstrating how the project could have been re-evaluated if the new early support tools had been applied.
The creation of a valid, early decision-making tool was the second purpose of the present Nordic joint effort.
In the era of digitalization and as the importance of innovative technology expand in the health care sector, new practises is constantly evolving.New technology enables refinement and personalisation of existing health care practice, which can potentially result in preventing chronic illness diseases and saving more lives [24].But although the technological revolution within health care shows great potential, not all technological innovation serve their purpose.Documenting the effects of health care innovation is therefore essential in dealing with prioritizing adequate technology implementation.Early value assessments and decision making becomes more urgent when value-based health is an over-arching goal.Further, the belief in a technology-induced shift to solve all challenges introduces misconceptions.To achieve value of new technology in the era of digitalization, organizational innovation is required.That is why managerial decision support is vital for health innovations to diffuse [25].Most new technologies are technology-enabling services, not products.When we only evaluate the technology and not the ecosystem around, we are unable to achieve the value of the new service.Another misconception is that for services to diffuse, the innovation must fit into the workflow and provide meaningful benefit.Thus evaluation should take place within the intended setting.Public sector actors cannot maintain cutting edge know-how in all the different areas needed to innovate, but must rely on cooperation with suppliers to succeed [26].When early decision-making support partner with innovation, it provides an opportunity to document the value of new services.
The present decision support tool and user guide represents a first attempt to share practice.The next phase is to test the tools within healthcare to provide decision support and share experiences, between institutions and in collaboration with the health care industry.Although, this is an important first step in developing a standardized tool for assessment of health innovation in the Nordic region of Europe, the tools and the user guide are still to be tested in real world situations.Research is therefore needed to validate the usability and the precision of the decision support tools and to suggest adaptions such that the tools are applicable in all participating Nordic countries.

Conclusions
NoTeB was successful in uniting Nordic countries in a common objective to develop a decision-making tool and a user guide to assess health innovations.Although the tools and the guide are still to be tested, this is a first step in developing a standardized tool for innovation assessment among the Nordic countries.

Figure 1 .
Figure 1.Summary of workshop exploring needs and aligning on content specification for the development of the user guide "How to assess value and benefits of innovation".

Figure 2 .
Figure 2. The User guide "How to assess value and benefits of innovation", the end-product of the Nordic cocreation process on decision support tool.

Figure 3 .
Figure 3.The four benefit domains in value assessments, description of each domain, its eight categories and examples of measurement tools in each domain.The domains are HTA categories of each domain and measurement tools.

Figure 4 .
Figure 4. Overview of suggested decision support methods in the various innovation stages as presented in the user guide "How to assess value and benefits of innovation".
Both the decision support tools and the user guide are developed based on discussions between the Nordic partners in NoTeB, where past experience and theoretical input was the fundament for the work process.
The project participants include Innovation Skåne, Innovation Akademiska at Uppsala University Hospital, Aalborg University Hospital at North Denmark Region, Centre for Connected Care (C3) at Oslo University Hospital, Oulu Health Labs at Oulu University Hospital, Business Oulu and Centre for Health and Technology at University of Oulu.There are two main aims for each partner in the Noteb project: 1) common testing among the Nordic countries, and 2) develop and agree upon an early assessment framework in health care.The present study addresses aim two.