The Application of Data Envelopment Analysis in Healthcare Performance Evaluation of Rehabilitation Departments in Hungary

Purpose: Performance evaluation is a general problem both in production and service systems. Generally, operation performance is determined based on input resource utilization and on outputs related data. Performance evaluation is especially complicated when both financial and nonfinancial indicators must be considered in the evaluation of the efficiency of healthcare system. The purpose of this paper is to apply data envelopment analysis (DEA) in order to measure the efficiency of rehabilitation departments curing musculoskeletal diseases. Methodology/Approach: The evaluation of the efficiency of rehabilitation departments includes several parameters. Performance evaluation becomes complicated when several evaluation criteria must be taken into consideration at the same time. In these cases, scoring methods are generally used, which transform performance data into a common scale and an aggregate score is calculated with subjective weights. Using DEA the subjective element of evaluation is eliminated when the weights of inputs and outputs are determined. Findings: The applied DEA model evaluates the performance of rehabilitation departments. The presented analysis highlights the differences between the efficiency of the studied departments, and explores inefficiencies related to economies of scale. The slack values directly show the operational shortcomings in specific areas, and indicate the exact amount of the required changes. Research Limitation/implication: The applied DEA model evaluates the performance of rehabilitation departments. The presented analysis highlights the differences between the efficiency of the studied departments, and explores inefficiencies related to economies of scale. The slack values directly show the operational shortcomings in specific areas, and indicate the exact amount of the required changes. QUALITY INNOVATION PROSPERITY / KVALITA INOVÁCIA PROSPERITA 21/3 – 2017 ISSN 1335-1745 (print) ISSN 1338-984X (online) 128 Originality/Value of paper: The originality of the paper lies on the identification of inputs and outputs for the applied DEA model as only nonfinancial indicators were taken into consideration. The analysis involves all rehabilitation departments of the Hungarian healthcare system; consequently, conclusions related to the general state of this area can be drawn. Category: Research paper


INTRODUCTION
Quality has now become a key factor of competitiveness that cannot be disregarded by any service sector, including healthcare.The meaning and dimensions of quality are interpreted in different ways in the health sector.In its 1989 report, the WHO Working Group on Quality Assurance identified four components of quality: performance (as technical quality), efficiency of resource use, patient satisfaction, and risk management.Donabedian (1980) highlights efficiency as a separate dimension of quality.Despite the different interpretation, there is consensus that the concept of quality cannot be separated from that of efficiency.
In order to improve the operation of any system it is important to measure operational related characteristics and to evaluate system performance.Performance evaluation and efficiency measurement of health systems have several similar purposes.According to Vitrai and Vokó (2012), the purpose of efficiency analysis is to improve the health of the population, to share the financial burdens of health deterioration and to meet the expectations of the citizens.
Performance evaluation is particularly complicated, when several conflictive evaluation criteria must be considered at the same time.Evaluation is especially difficult, when these criteria are measured on different scales.Data envelopment analysis (DEA) is a special type of scoring method which can handle this problem.DEA has become commonly used in many service areas; it has been used, for example, to measure the efficiency of bank branches (Sherman, 1984), restaurants (Reynolds and Thompson, 2007), hotels (Hwang and Chang, 2003) and hospitals (Rosko, 1990).An interesting and new application area of DEA is the evaluation of the performance of participants in business simulation games (Koltai, et al, 2017).
The objective of the paper is to compare musculoskeletal rehabilitation departments in the field of in-patient care in Hungary using DEA.The paper presents the characteristics of Hungarian healthcare and suggests efficiency improvement possibilities based on DEA results.
In the following part of this paper first, some basic DEA models are introduced.Next, the most important characteristics and conditions for evaluating musculoskeletal rehabilitation departments are discussed.Finally, the DEA results are presented, some important conclusions are drawn, and further research possibilities are outlined.

BASIC CONCEPTS OF DATA ENVELOPMENT ANALYSIS (DEA)
Data envelopment analysis (DEA) is a quantitative method used to compare the performance of production and/or service systems.These systems are called as decision making units (DMU).The objective of DEA is to determine the efficiency of decision making units relative to each other.The best operation performance is determined based on the input and output data of the analyzed decision making units.
Various models are used depending on the application environment and on the objectives of management.If the objective is to minimize the amount of inputs without the change of output values, than an input-oriented model should be applied.If the objective is, however, to maximize the amount of outputs without the change of input values, than output-oriented models should be used.
Different models are applied when the size of the DMUs is an important factor in the analysis.We may consider constant return to scale (CRS) or variable return to scale (VRS) models.If we assume a constant return to scale (CRS) relationship between the input and output values then the size of the input does not influence the marginal change of output.When the effect of the change of input is not constant then a variable return to scale (VRS) relationship exists.
Several DEA models can be applied depending on the application environment.
The selection of the model is influenced, for example, by the characteristics of the DMUs, by the nature of available data, and by the evaluation criteria.We applied an output-oriented variable return to scale slack-based model for the evaluation of rehabilitation departments.Notations applied in this paper are listed in Tab. 1. DEA is based on the idea that the measure of output is always less than the measure of input used to generate the output.Thus, the ratio of output measure and input measure must be less than or equal to 1 (Charnes, Cooper and Rhodes, 1978).The input-oriented CRS model describing these constraints and goals is the following: We applied the output-oriented variable return to scale version of the basic model (1).In case of output-oriented models, relative efficiency is measured by the ratio of the weighted inputs and weighted outputs.The value of this ratio is the reciprocal of the relative efficiency score of the input-oriented model, consequently, the objective function must be minimised.According to the variable return to scale (VRS) model the value of the weighted input is modified by a variable uiR.By definition, output-oriented relative efficiency is always greater than or equal to 1.The output oriented VRS model is as follows: (2) Linear programming (LP) problem (2) has an infinite number of solutions.Fixing the weighted output at value 1 and rearranging (2) by eliminating the ratio of variables, we get the primal version of the model.The dual version of problem (2), however, has more practical relevance.The output oriented VRS dual model is as follows: The optimal value of the objective function of (3) is η*.Models (2) and ( 3) are based on a radial measure of efficiency, that is, all outputs are increased proportionally by the same ratio (η*).The slack based model (SBM) proposed by Tone (2001) uses independent input/output changes.The difference of the actual value and the best possible value is called slack.In (4), sk + indicates the output increase possibility of output k, and si -indicates the input decrease possibility of input i.Based on the slack values the following efficiency measure can be calculated: 1 Objective function ( 4) is a non-oriented measure of efficiency.Depending on the orientation of the analysis either the nominator or the denominator can be ignored in the objective function.The output oriented approach applied in the following part of the paper uses the following objective function: (5) In the following, first, the importance of DEA in healthcare based on some relevant literature sources is summarized.Next, the application of DEA for the analysis of musculoskeletal rehabilitation departments in Hungary is presented.

APPLICATION OF DEA IN HEALTHCARE
The practical application of DEA is becoming more common in the field of healthcare.The operation of health systems shows differences by the effect of external and internal environmental factors.External environmental factors are, for example, demographic characteristics, financing, human resource trends, and regulations.Internal environmental factors are, for example, hospital capacity, facilities and amenities, technology, healthcare delivery and ownership structure.As a consequence of the different characteristics of health systems the objective of the analysis and the input and output variables applied can be different in each case.When analysing healthcare efficiency, several international studies focus on economic and social factors, and lifestyle such as unemployment rate or level of education (see, e.g., Spinks and Hollingsworth, 2009).Others use financial indicators because healthcare expenditure has important effect on public health, as well as on the operation of health systems (Hadad, Hadad and Simon-Tuval, 2011;Portafke, 2010;Rivera, 2010;Schoenberg, et al., 2007;Asandului, Roman and Puiu, 2014).
According to Akazili, et al. (2008), DEA is an essential method for evaluating the efficiency of health systems.The strategic and operational aspects of resource management can be evaluated by DEA (Akazili, et al., 2008).Kirigia and Boussofiane said that DEA enables to measure the efficiency of operational strategies in the health sector.As a result, the performance and the reference set of health systems can be determined and the efficiency of resource allocation can be evaluated (Kirigia, Emrouznejad and Sambo, 2002;Boussofiane, Dyson and Thanassoulis, 1991).
The selection and use of variables are especially difficult in Hungary as a consequence of incomplete data collection and data management in the health system.The use of DEA model in the Hungarian health sector has been limited to only a few studies (see, Csákvári, et al., 2014).

BACKGROUND OF THE RESEARCH
The objective of our research is to determine the operational efficiency of the musculoskeletal rehabilitation departments in Hungary.The aim of medical rehabilitation is to stabilize and restore the physical and mental conditions of patients, to reduce the harmful consequences of disability, and to facilitate the social reintegration of patients.In Hungary, rehabilitation is running in hospital departments, in specialist outpatient clinics and in specialist practice (inpatient), where each rehabilitation department offers a different service: musculoskeletal, cardiac, psychiatric, pediatric, internal medicine and pulmonary rehabilitation.Departments show differences in organizational structure, financing, and the condition and treatment of patients, therefore, it is appropriate to analyze them separately.
The number of musculoskeletal disease patients is rapidly growing (Vos, Murray and Barber, 2015).The increasing number of musculoskeletal disease patients in the EU and in highly developed health-culture countries is a leading health problem.Musculoskeletal diseases have impact on the quality of life as well as on working and life expectancy.The treatment of these diseases imposes socialand financial-bourden on healthcare, on society, on patients and on their family (Dénes, 2015).Consequently, the rehabilitation of these diseases requires more attention.
The musculoskeletal rehabilitation process consists of different activities with shorter and longer periods of time.These methods and activities are such as diagnostics, physiotherapy, speech therapy, psychological care and training of the use of medical devices.Rehabilitation medicine makes differences between post-acute rehabilitation and planned rehabilitation.In the case of acute diseases or trauma, and when a chronic disease is getting worst (unexpected complications of an existing disability) post-acute rehabilitation is needed.Postacute rehabilitation activities should start immediately after the need of healthcare has arisen, or within a maximum of one month.In the case of the planned rehabilitation the starting point of the treatment is not the most important factor.
In this paper we analyze the efficiency of inpatient departments of musculoskeletal rehabilitation in Hungary.The data used in this study are based on the 2014 national annual survey of the National Statistical Data Collection Program.International and national studies, data on population health and characteristics of the Hungarian health system are also considered in the analysis.Accordingly, different rehabilitation departments of the same institution have been aggregated when largely or entirely, they have shared common human resources.Hence, the number of organizational units has decreased from 116 to 87 departments.As a consequence of data collection problems 7 more departments have been ignored.Thus the final number of analyzed rehabilitation departments has been reduced to 80.
The efficiency evaluation of the 80 musculoskeletal rehabilitation departments and the results of the detailed analysis are presented in the following section.

THE MAIN RESULTS OF RESEARCH
The 80 rehabilitation departments are considered as the decision making units (DMU) in the analysis.Two outputs and four inputs were considered in the analysis.The two outputs are defined as follows: • number of patients' day: total number of days, musculoskeletal patients stay in the department and undergoing rehabilitation procedures; • number of patients discharged: total number of musculoskeletal patients who leave the department because of the rehabilitation treatment is finished or because of transfer to other department.
The four inputs represent the most important resources applied by the departments, which are the following: • number of hospital beds: this indicator provides a measure of the resources available for delivering services to musculoskeletal patients at the department; • number of physicians employed: the full time equivalent (FTE) of the number of doctors at the rehabilitation department (part time doctors with joint affiliation are considered with a 0.3 weight); • number of nurses employed: the full time equivalent (FTE) of the number of nurses at the rehabilitation department; • number of professional healthcare workers and other non-physician specialist (psychologists, speech therapists, physiotherapists, qualified masseurs, conductive teachers, physiotherapist assistants, occupational therapists, orthopaedic technician, social assistant, medical physical education, special education teachers, dieticians, other therapists, others).
As a consequence of the nature of the different forms of employment (part-time, full-time) we recommended the modification of the number of employees by using weights.These weights express the number of hours the employee is involved in the activity of the department.
We also note, that in order to avoid numerical problems when large linear programming model is solved, data must be scaled.In this case expressing the number of patient's days in thousand days, that is, dividing this data by 1,000 solves scaling problems.
Some statistical data describing the characteristics of input and output data are summarized in Tab. 2. Differences in size of rehabilitation departments can be assumed based on the large differences between minimum and maximum values, and on the value of standard deviations.
Our analysis consists of two parts.First, scale efficiency using output-oriented radial models based on problem (3) is analyzed.Next, the efficiency and improvement possibilities of each department using output-oriented variable return to scale slack-based models based on problem (5) are determined.

Analysis of the Size of Rehabilitation Departments
Output-oriented radial models are applied to explore the differences in size of the 80 rehabilitation departments.15 departments (19%) show constant return to scale (CRS) characteristics.The size of these departments can be considered optimal.These departments are almost always located in spa towns and many of them are operated by specialized hospitals.
10% of the departments (8 departments) can be considered oversized, as a consequence of decreasing return to scale (DRS) characteristics.Many of these departments are aggregated and operated by hospital in large cities.The technical efficiency of four of these departments is 100%.Although they are oversized, their operation is efficient.In the case of the other 4 departments large size is paired with operational problems.
57 departments (71%) can be considered too small as a consequence of the increasing return to scale (IRS) characteristics.These departments are operated by small specialized hospitals in Budapest or in small towns in rural areas.Among the departments with IRS characteristics, 10 departments operate efficiently despite their small size, but the operation of the other 47 departments is inefficient.
The different size properties of the departments indicate the special characteristics of the Hungarian musculoskeletal rehabilitation system.Most of the departments with CRS are located in spa towns where planned and self-care patients are being cured (e.g., arthritis, osteoarthritis).In the case of planned rehabilitation, the number of beds and the utilization of human resources can be accurately planned.
Most of the units with DRS property are aggregated departments thus they operate high number of active beds and human resources.
The high number of the IRS departments indicates the regional and demographic characteristics of the Hungarian health system.Some of these small-sized departments are located relatively close to each other and concentrated in Budapest, where there are a large number of patients.Other small departments operate in small towns.

Analysis of Improvement Possibilities
Output-oriented variable return to scale slack-based models were used to explore operational shortcomings, and improvement possibilities.Tab. 3 summarizes the input and output data and SBM efficiency score of some typical departments.The efficiency values of the 80 departments are between 0.47 and 1, but most of the scores are between 0.5 and 0.6, and between 0.9 and 1.The number of efficient (η = 1) departments is 25, although they do not have the highest output (lowest input) quantity.Each of these groups applied different strategy to operate their systems successfully.
Column 2 shows the number of hospital beds.We can see that the highest number of hospital beds is found at department H51, and the efficiency score of this department is not the highest.
Department H77 is similar to H51 in size, but it operates efficiently.Similar conclusion can be drawn in connection with department H57.This department has similar size characteristics as H8.H57 uses almost the same number of hospital beds as H8, even though the operation of H57 is inefficient.The reason for this is that department H57 should have generated higher outputs based on its inputs.
Columns 3-5 show the number of employees.The highest number of employee is found at department H83.Although H83 has similar number of employees as H35, the efficiency score of H83 is lower than that of H35.At department H83 the high number of employees was not pared with high values of the outputs.
Column 7 shows the number of patients discharged.Departments H35, H51 and H77 are among the departments with the highest value, with higher than 3,000 number of patients discharged.Despite of the fact that these departments achieved high output values by using high number of beds and employees, not all of them are efficient.Department H51 is inefficient with a 0.805 efficiency score.
The results show that we cannot draw conclusions on efficiency based exclusively on the amount of inputs or outputs.Only a combined analysis of all the inputs and outputs can provide information about the efficiency of operation and about the possible improvements of efficiency.
DEA provides information about efficiency values, and also about improvement possibilities.The last column of Tab. 3 shows the reference sets of inefficient departments.Following the strategy and operational policy of reference departments, inefficient departments can achieve efficient operation.As Tab. 3 shows, inefficient department H11 should adopt the good practices of H8 and H79.
The detailed analysis of the results revealed that most of the reference departments are located in spa towns, in small towns and/or those with mixed profile.For example, H8 is reference department of 35 inefficient departments.H8 is a small-sized department with mixed profile, and it is a reference department of several similarly small-sized departments.
Department H39 is an interesting example.This department is efficient, but does not appear in the reference set of any other departments.H39 is located in a new facility and carries out programmed rehabilitation activities.Similarly, department H33 uses unique strategy; it is one of the departments belonging to the 4 university hospitals in Hungary, and its operation, organizational structure, and management are different from that of the other hospitals.
The evaluation of the reference sets, however, must be made with care.It can be possible that an efficient department is in the reference set of an inefficient department which, however, has a different profile; consequently its operation practices cannot be implemented.Department H37, for example, has a mixed profile and, therefore, carries out various rehabilitation activities.This department is the reference set of department H63 which is a neuro-stroke rehabilitation department carrying out a specific activity.A similar conclusion can be drawn in connection with the departments H8 and H11.
The efficiency score in Tab. 3 is an aggregate measure of operational efficiency.The independent change of each output and input are expressed by the slack variables.Slack values of some typical departments are summarized in Tab. 4. The results show that most of the inefficient departments have shortcomings with respect to the number of patients discharged.In the case of department H11, for example, the number of patients discharged should be increased by 104.1 in order to achieve efficient operation, independently from the number of completed days.According to the number of completed days, output shortage appears in only 5 cases.One of the 5 departments is H54 which can be seen in Tab. 4. For that matter, the days of treatment cannot be changed, since different types of diseases require different time for treatment, which is frequently determined by professional protocols.There are, however, subjective elements of the evaluation of the period of treatment.Consequently, there may be organizational measures which can help to reduce the number of treatment days.
Analysis of the slack values of the number of beds (si -) show that one non-zero value can be found (none in Tab. 4).We can draw misleading conclusion from these values.A 0 slack values may indicate, that there are no improvement possibilities, that is, in the Hungarian rehabilitation departments the decrease of the number of beds does not increase efficiency.This, however, is not necessarily true.Since output-oriented models are used, the input slack values are not maximum values.In this case, the zero input slack does not necessarily mean that the input cannot be reduced.Nevertheless, a high input slack value indicates that an excess use of input has occurred.According to the slack values we can certainly conclude that in the case of inefficiency departments H11 and H32 the number of nurses can be significantly reduced.Similarly, in the case of departments H51 and H54, the number of healthcare workers is considerably higher than necessary.

CONCLUSION
The application of DEA is not widespread in the healthcare system in Hungary; hence, the presented analysis can be an important example, which illustrates the potential benefits of its application.The presented study contains several simplifications.These are, however, due to the authors' efforts to present the possibilities of DEA using a relatively simple example, and not due the methodological deficiency of the applied method.
In the first phase of the analysis an output-oriented variable return to scale radial model has been used for analyzing the size of the departments.We concluded that the departments show significant differences in size, and the size of most of the departments is not optimal.Most of the units are smaller than the optimal size.
In the second phase of the study an output-oriented variable return to scale slackbased model has been used for exploring the improvement possibilities of the rehabilitation departments.We showed how operational shortcomings can be identified.The results indicate the problems of inefficient departments and the possible improvement of some seemingly well-operating departments as well.
The evaluation of reference sets can provide additional useful information to improve operation.The reference set contains those departments whose operational practice should be followed to improve efficiency.
One of the main shortcomings of the presented analysis is that the mix of patients is not identical in all departments.Despite of the fact that all departments perform musculoskeletal rehabilitation, not all of them perform exactly the same activity.Most of the departments have a mixed profile, but the ratio of the rehabilitation reasons is not identical, furthermore, there are specialized institutions as well.Rehabilitation causes requiring a more complex treatment (for example brain injury) have a different input need than rehabilitation after simple surgeries.The objective of our further research is to eliminate this problem by categorizing the institutions according to their profile characteristics and using DEA models with categorical variables.
Another shortcoming of the presented study is that quality and patient satisfaction are not considered among the outputs.There is no theoretical obstacle to take into account such outputs.In order to incorporate quality oriented outputs, however, reliable quality/satisfaction related data must be determined for all the departments.The lack of such data is an important shortcoming of the Hungarian healthcare system.

Table 1 -
List of Notation

Table 2 -
Statistical Data of Input and Output Data

Table 3 -
Input and Output Data, and SBM Efficiency Score of Some Typical Departments

Table 4 -
The Efficiency of Some Typical SBM,Input and Output Slack Values