Implementing an ITIL-based IT Service Management Measurement System

Antti Lahtela and Marko JanttiSchool of Computing

University of Eastern FinlandP.O.Box 1627, FIN-70211 Kuopio, Finland

firstname.lastname@uef.fi

Jukka KaukolaTieto, Energy Finland

P.O.Box 1199, FIN-70211 Kuopio, Finlandjukka.kaukola@tieto.com

Abstract—The IT Infrastructure Library (ITIL) providesguidelines for IT service providers how to design, manage andsupport IT services. ITIL is the most widely used IT servicemanagement framework. It consists of best practices that canbe used in implementing, for example service support processes,such as incident management and problem management. Al-though ITIL includes a wide list of process metrics, it doesnot provide sufficient information how IT service managementprocess measurements should be done in practice. The researchproblem in this paper is how the measurement of the ITservice support processes can be improved? In this paper,an ITIL-based IT service management measurement system(ITSM-MS) and its implementation project are presented.The ITSM-MS can be used to measure the performance ofIT service support processes. The ITSM-MS was developedin cooperation between MaISSI (Managing IT Services andService Implementation) research project and an IT servicecompany in Finland.

Keywords-IT Service Management; ITIL; Measurement.

I. INTRODUCTION

Service measurement plays an important role in IT ser-vice management that is one of the subfields of ServicesComputing science [1]. Many IT service organizations con-sider the measurement of IT service management processes,especially service support processes, as a difficult task.Difficulties are mainly due to the following four reasons:1) IT organizations do not have a structured approach formeasuring IT services and service management processes,2) tools used by service support teams do not enableeffective measurement, 3) IT service management standardsand frameworks do not provide practical examples howto measure support processes, and 4) there are too manyoptions what to measure in service management.

First, IT organizations need a structured approach formeasuring IT service support processes. The IT Infrastruc-ture Library (ITIL) is the most widely used frameworkfor IT service management. The support processes of ITILinclude incident management, problem management, changemanagement, configuration management, and release man-agement. In this paper, we focus on the first two processesthat are often called front-end support processes [2]. Themain objective of incident management process is to dealwith all incidents including failures, questions and queries

reported by the users [3]. Problem management in turnfocuses on preventing problems and incidents, eliminatingrecurring incidents and minimizing the impact of incidents.ITIL version 2 contains very little information regardingthe measurement of service support processes [4]. TheContinual Service Improvement (CSI) of the ITIL version 3proposes a 7-step improvement process to support IT servicemanagement measurement activities [5]. In the CSI, the mea-surement is based on three basic concepts: critical successfactors (for example, reducing IT costs), key performanceindicators (for example, 10 percent reduction in the costs ofhandling printer-related incidents) and metrics (for example,cost of the improvement effort). Unfortunately, the 7-stepimprovement program seems to be very abstract and difficultto adopt in practice.

Second, service support processes are very tool-orientedprocesses. Unfortunately often, the service reporting andmeasurement functions are the weakest parts of servicedesk tools. The tool should enable effective measurementof both resolution times and volumes of support requests.If the service reporting and measurement function doesnot work properly, it will remarkably increase the manualwork in producing process performance reports. In many ITorganizations, process managers still have to use MicrosoftExcel to produce monthly reports regarding customer sup-port processes.

Third, IT service management standards and frameworksshould provide process managers with more practical exam-ples how to measure support processes. Although, ITIL hasintroduced a good selection of IT service support metricsboth in version 2 [4] and version 3 [3], it does not providesufficient information how IT service management processmeasurements should be done in practice. ITIL version 3incident management process contains 15 potential metricsfor incident management (for instance, total numbers ofincidents, number and percentage of major incidents, meanelapsed time to achieve incident resolution) and 10 metricsfor problem management (for example, the total numberof problems/period, the percentage of problems resolvedwithin SLA targets, the average cost of handling a problem,the number of major problems). Instead of long lists ofmetrics, IT service management frameworks should provide

2010 Fourth International Conference on Digital Society

978-0-7695-3953-9/10 $26.00 © 2010 IEEE

DOI 10.1109/ICDS.2010.48

255

2010 Fourth International Conference on Digital Society

978-0-7695-3953-9/10 $26.00 © 2010 IEEE

DOI 10.1109/ICDS.2010.48

249

more practical examples how to use metrics. Additionally,metrics could be divided by priority into primary metricsand secondary metrics.

Fourth, IT organizations can measure their IT operationsfrom many different perspectives. In software engineering,much has been written about how to measure software qual-ity. Marik et al. [6] have examined software validation andverification metrics from the testing viewpoint. Accordingto Roche [7] software quality measurement requires a struc-tured framework because measurement is a very error proneprocess. Similarly, Hall and Fenton [8] emphasize that themeasurement program should always have clear and specificgoals. Boehm and Basili [9] have stated that 80 % of thedefects come from 20 % of the modules and about half of themodules are defect free. Thus, measurement program shouldaim to identify the weakest components in the IT infrastruc-ture. Software quality metrics can be divided, for example,into efficiency metrics (transaction time), correctness metrics(complexity, MTBF), reliability metrics (down times), andmaintainability metrics (number of modules, number of er-rors per unit) [10]. IEEE Standard Dictionary of Measures toProduce Reliable Software [11] divides metrics simply intoprocess metrics and product metrics where product measuresare applied to software objects and process measures areapplied to the activities of software development, test andmaintenance. In IT service management, IT organizationshave also several measurement targets to choose. They canmeasure the performance of any IT service managementprocess, the maturity of IT service management processes oran IT organization (for example, using CMMI for services[12] or other maturity assessment models [13]), customersatisfaction on IT services and processes, service businessperformance (for example, costs of service unavailability),and the quality of the software or service (number of errorsrelate to a configuration item). Naturally, each measurementtarget requires a different type of measurement approach.

Besides ITIL, there are several other IT service manage-ment standards and frameworks that address the need formonitoring and measuring service management and providetheir own set of metrics. Control Objectives for Informationand related Technology (COBIT) framework [14] is designedfor IT governance purposes. COBIT provides both processmetrics and maturity level metrics for each delivery andsupport (DS) process, such as DS8 Manage Service Deskand Incidents and Manage Problems. Examples of metricsinclude first-line resolution rate, % of incidents reopened, %of problems recorded and tracked, and % of problems thatrecur (within a time period) by severity. ISO 20 000 is ITIL-compliant auditable standard for IT service management thatconsists of two parts: specification for service management[15] and code of practice for service management [16].One of its requirements is that the organization shall applysuitable methods for monitoring and measurement of theservice management processes. ISO 20 000 also requires

that the organizations produces reactive reports, proactivereports, and scheduled reports regarding IT service manage-ment activities. However, ISO 20 000 does not tell whichmetrics should be used to measure the processes.

There seems to be a clear need for better IT servicemanagement measurement frameworks that would define thegoals, roles, activities and would be easy to adopt in practice.Additionally, support process managers could benefit fromsome existing software quality measurement frameworks,such as the Defect Management Framework of Quality assur-ance Institute [17], and the Software Quality Measurementa Framework for Counting Problems and Defects [18]

A. Our Contribution

This case study is a part of the results of MaISSI(Managing IT Services and Service Implementation) re-search project at the University of Eastern Finland, Schoolof Computing, ISSE (Information Systems and SoftwareEngineering) research group, Finland. The main objectivesof the MaISSI project are to enhance the introduction andimprovement of IT service management processes, supportthe development of new services and high quality software,and share new ideas and experiences regarding customersupport in the network of organizations. The work in ourresearch project has been divided into eight subprojects(MaISSI pilot projects). The pilot project of the IT servicemanagement measurement system is one of the pilot projectsand it was carried out during years 2008 - 2009. The maincontribution of this paper is to:

• describe the implementation process of the ITIL-basedIT service management measurement system (ITSM-MS) and

• describe the system architecture and the main functionsof the ITSM-MS.

The remainder of the paper is organized as follows.In Section 2, the research questions and methods of thisstudy are described. In Section 3, we describe how theITSM-MS was implemented. Section 4 describes the systemarchitecture and the main functions of the ITSM-MS. Thediscussion and the conclusions are given in Section 5.

II. RESEARCH QUESTIONS AND METHODS

The main research problem of this study is how themeasurement of the IT service support processes can beimproved? The research problem was addressed by theapplication service manager of the case organization. Bothconstructive, case study methods [19] and action researchmethods were used in this study.

A. Case Organization

Our case organization is a medium size business unit(around 120 employees) of a bigger organization that is oneof the leading IT service companies in Northern Europewith over 16 000 employees. The company provides IT,

256250

R&D and consulting services for various industries, suchas banking and insurance, energy, telecom and media, andhealthcare. Our pilot was implemented together with the caseorganization and the MaISSI research team. Improvement ofmeasurement activities was considered as a very importantimprovement target in the case organization and was selectedto the main topic for the pilot project.

IT service support processes, such as incident manage-ment, problem management are part of the case organiza-tion’s business framework WayToExcellence (W2E). Servicedesk acts as a single point of contact for customers andusers. The service desk assigns the incident to the backoffice (second-line support) which in turn assigns the caseto the product support team, if necessary. The organizationuses a java-based tool for handling all the incidents. Theincident management tool supports the ITIL-based servicemanagement processes. The organization has been quitesatisfied with the tool functions and its configuration options.However, they stated that producing measurement reportsregarding the performance of service support processes isnot effective.

The pilot project between the MaISSI research team andthe case organization was carried out between years 2008and 2009. The main goal of the pilot project was to establisha system that enables better measurement of IT servicesupport processes. The process included the following steps:

• 8 August 2008: The kickoff meeting of the pilotproject.

• 1 September 2008: The 1st requirement specificationmeeting.

• 6 October 2008: The 2nd requirement specificationmeeting.

• 23 October 2008: The 3rd requirement specificationmeeting.

• October - November 2008: The design phase of theITSM-MS.

• January - April 2009: The implementation phase ofthe system.

• 9 April 2009: The final review meeting of the system.• April - May 2009: Trainings in the case organization.• May 2009 - July 2009 : Initialization of the system.

III. ESTABLISHING ITSM-MS

In this section, we will introduce how the ITSM-MS wasestablished between the case organization and the MaISSIresearch team. The work was divided into three phases:the requirements specification phase, the design phase andthe implementation phase. The kickoff meeting of the pilotproject was arranged in 8th August 2008. In that meeting,the representatives (the application service manager and theIT service manager) of the case organization reported thatthey have problems in measurement and reporting activitiesregarding IT service support processes. Additionally, thegoals for the pilot project were specified.

A. Requirements Specification Phase

Few weeks after the kickoff meeting, the 1st requirementspecification meeting for the ITSM-MS was arranged and thefirst requirements for the system were defined. Requirementswere gathered together with the application service managerand the IT service manager of the case organization. Thefollowing general requirements were identified:

• The system must provide real-time measurement infor-mation about IT service support processes

• The system must be easy to use (produced measurementreports must be clear and easy to understand).

• The system must support ITIL-based IT service supportprocesses, especially incident management and problemmanagement.

• The system must provide a dynamic user interface(for example the user can select different values forproducing graphs).

• The system must be implemented with the technologyprovided by the case organization.

Additionally, the MaISSI research team introduced someexamples of IT service support metrics that ITIL provides. Inthe 2nd and the 3rd requirement specification meetings, therequirements for the ITSM-MS were clarified. The conceptsin the requirements specification were based on the caseorganization’s service management framework and the ITILframework.

B. Design Phase

The design phase of the system was carried out duringOctober - November 2008 and it was divided into two stages.The first stage (October 2008) was performed as an actionresearch where one researcher from the MaISSI projectteam was working intensively in the case organization. Theresearcher took part in the case organization’s IT servicemanagement team and collected data from the incidentmanagement tool. The second stage (November 2008) aimedat preparation of the system specification based on the datathat was collected earlier. As a result of the design phase,the system specification was documented and reviewed at24th November 2008.

C. Implementation Phase

The implementation phase of the ITSM-MS was executedin January - April 2009. This phase was also performedas an action research where one member of the MaISSIresearch project team worked in the case organization’sfacilities. Based on the system specification, the ITSM-MSwas implemented with the help of the case organization andits employees. The final review meeting of the system wasarranged in 9th April 2009. Participants at that meeting werethe application service manager and the IT service managerof the case organization and the MaISSI research projectteam. After this, the system was deployed.

257251

D. Training and the Initialization Phase

Trainings for the employees were carried out during April- May 2009 by the application service manager of the caseorganization. After trainings, the initialization phase of thesystem was performed and now the ITSM-MS is currentlyin use in the case organization.

IV. SYSTEM ARCHITECTURE AND MAIN FUNCTIONS

In this pilot project, the ITSM-MS was constructed for theinternal use of the case organization and its employees. Thesystem was targeted to process managers, product managersand project managers to work as a real-time measurementtool that is connected to every IT service support process.From the technical point of view, the ITSM-MS was imple-mented using Microsoft .NET environment, Microsoft VisualStudio, C# programming language and Oracle database. TheITSM-MS uses the database of the incident managementtool. Microsoft Visual Studio was used to implement the userinterface and the system functions while Oracle databasestores all the data from the IT service support processes.

Figure 1 shows the general system architecture of theITSM-MS. The service desk (SD) of the case organizationreceives an incident from the customer, a ticket is enteredinto the incident management tool and a new case is opened.The case organization uses ITIL-based IT service supportprocesses for resolving the case and all the data that isused during these processes is documented into the incidentmanagement tool. When the case is resolved, it is sent backto the customer and the case is closed.

The ITSM-MS uses the data that is documented in theincident management tool. The user can use the system fromthe case organization’s Intranet and make real-time graphsabout different cases. The user’s input is transformed into anSQL-query which returns a result from the database. Basedonto this result, the ITSM-MS draws a graph of the metricinto the user’s computer screen in the Intranet.

The user interface of the ITSM-MS contains differentvalues and functions that the user must select for generatingthe graph and saving the user profile. The user profile islater used for generating the graph based on saved valuesand functions. In that way, the user doesn’t have to enterall the values again for the graph. The user can select aspecified value or multiple values depending on what typeof graph is needed to represent (Figure 2):

• Customer: All customers of the case organization.• Product: All products of the case organization.• Type: Different types of cases (for example incident,

problem, change request or known error).• Project: All projects of the case organization• Service Desk: SD of the case organization.• Assigned to: The person who is responsible of the case.• Classification: All cases are classified (for example

error in program, hardware problem or error in doc-umentation).

Figure 1. The system architecture of the ITSM-MS.

• Impact: The impact of the case (for example standardimpact, major impact or note).

• Business impact: How case affects to the service orthe business (for example no business impact and minorerror or esthetic).

• Priority: Priorities of cases: 0 - undefined, 1 - urgent,2 - high, 3 - medium and 4 - low.

• Cases: Some particular cases.• Group by: Days, weeks, years.• Graph: The graph of the metric can be presented as a

line graph or a bar graph.• Point labels:Yes or no.• Graph between days: The starting point and the

ending point for the graph.• Search profile: Search for a particular saved user

profile.• Save search profile: Save the current user profile.• Target limits: This is used for the traffic lights (ex-

plained later in this paper).

The user of the ITSM-MS can select target limits forthe metric and save them to a user profile. Limits are usedto show the current situation of different cases by trafficlights that are showed in Figure 3. The pointer on the greensection means that the situation is fine and there aren’t somany unsolved cases. The pointer on the yellow sectionmeans that the situation needs more attention and there arefew unsolved or non-closed cases to resolve. Finally, thepointer on the red section means that there are too manyunsolved cases in the incident database. This means that thecase organization must take action for resolving those casesand turn the pointer from the red section back to the greensection. Numbers below the traffic lights describe cases thatare currently open and the change on previous measuring

258252

Figure 2. The user interface of the ITSM-MS.

period, for example last month. Numbers above the trafficlights are the target limits.

The ITSM-MS can also be used as a miniature versionbesides of the full version that was showed in Figure 2.Figure 4 represents the mini window of the ITSM-MSsystem where a saved user profile is showed on the screen.The mini window can show all the saved user profiles thatthe user has created. Thus, the user doesn’t have to open thefull version of the ITSM-MS, if the required graph is savedon a profile. The mini window enables the faster use of theITSM-MS.

V. DISCUSSION AND CONCLUSION

This study aimed to answer the research question howthe measurement of the IT service support processes canbe improved? We used a technology-oriented approach toanswer the research question and implemented the ITIL-based IT service management measurement system (ITSM-MS) together with one business unit of the large IT serviceprovider organization in Finland. The system is noticed tobe a success in the case organization based on given andcollected feedback from the system users.

The main contribution of this study was to give a generalsystem overview of the ITSM-MS. The basic idea of the

Figure 3. Traffic lights.

259253

Figure 4. The mini window of the ITSM-MS.

ITSM-MS is that it provides real-time measurement infor-mation from IT service support processes. The ITSM-MScan be used by process managers, product managers andproject managers to monitor the performance of IT servicemanagement.

However, there are some limitations to this study. Thestudy was carried out in one business unit of the caseorganization. The requirements were gathered only from fewemployees of the case organization. The introduction phaseof the system was not performed during our pilot project.However, the application service manager and IT servicemanager informed that the ITSM-MS had been introducedto some process managers in the case organization. It hadreceived positive feedback.

Further research is needed to investigate the introductionof the systematic measurement process as a part of contin-uous service improvement. Additionally, we could collectfeedback from the users of the ITSM-MS and implementthe second version of the system.

ACKNOWLEDGMENT

This paper is a part of the research project MaISSI(Managing IT Services and Service Implementation) at theUniversity of Finland, Finland. MaISSI is funded by the Na-tional Technology Agency TEKES, the European RegionalDevelopment Fund (ERDF) and industrial partners.

REFERENCES

[1] L.-J. Zhang, J. Zhang, and H. Cai, Services Computing. Ts-inghua University Press, Beijing and Springer-Verlag GmbHBerlin Heidelberg, 2007.

[2] M. Kajko-Mattsson, “Maturity status within front-end supportorganisations,” in ICSE ’07: Proceedings of the 29th interna-tional conference on Software Engineering. Washington, DC,USA: IEEE Computer Society, 2007, pp. 652–663.

[3] Office of Government Commerce, ITIL Service Operation.The Stationary Office, UK, 2007.

[4] Office of Government Commerc, ITIL Service Support. TheStationary Office, UK, 2002.

[5] Office of Government Commerce, ITIL Continual ServiceImprovement. The Stationary Office, UK, 2007.

[6] V. Marik, L. Kral, and R. Marik, “Software testing & diagnos-tics: Theory & practice,” in SOFSEM ’00: Proceedings of the27th Conference on Current Trends in Theory and Practiceof Informatics. London, UK: Springer-Verlag, 2000, pp. 88–114.

[7] J. Roche, “Software metrics and measurement principles,”SIGSOFT Softw. Eng. Notes, vol. 19, no. 1, pp. 77–85, 1994.

[8] T. Hall and N. Fenton, “Implementing effective softwaremetrics programs,” IEEE Softw., vol. 14, no. 2, pp. 55–65,1997.

[9] B. Boehm and V. R. Basili, “Software defect reduction top10 list,” Computer, vol. 34, no. 1, pp. 135–137, 2001.

[10] E. Wallmueller, Software quality assurance: A practical ap-proach. Prentice Hall International, 1994.

[11] IEEE Standard 982.1-1988, IEEE Standard Dictionary ofMeasures to Produce Reliable Software. IEEE, 1988.

[12] F. Niessinka, V. Clerca, T. Tijdinka, and H. van Vliet, “Theit service capability maturity model version 1.0,” CIBITConsultants&Vrije Universiteit, 2005.

[13] IT Service Management Forum, “Itil service management selfassessment,”http://www.itsmf.com/bestpractice/selfassessment.asp,5 2007.

[14] COBIT 4.0, Control Objectives for Information and relatedTechnology: COBIT 4.0. IT Governance Institute, 2005.

[15] ISO/IEC 20000, IT Service Management, Part 1: Specificationfor service management. ISO/IEC JTC1/SC7 Secretariat,2005.

[16] ISOIEC 20000, IT Service Management, Part 2: Code ofpractice for service management. ISO/IEC JTC1/SC7 Sec-retariat, 2005.

[17] Quality Assurance Institute, “A software defect managementprocess,” Research Report number 8, 1995.

[18] W. Florac, “Software quality measurement a framework forcounting problems and defects,” Technical Report CMU/SEI-92-TR-22, 1992.

[19] R. Yin, Case Study Research: Design and Methods. BeverlyHills, CA: Sage Publishing, 1994.

260254