Journal of Industrial Engineering and Management

A design of evaluation method for SaaS in cloud computing

Chekfoung Tan¹, Kecheng Liu^1,2, Lily Sun³

¹Informatics Research Centre, University of Reading (United Kingdom)

²School of Information Management and Engineering, Shanghai University of Finance and Economics (China)

³School of Systems Engineering, University of Reading (United Kingdom)

 

Received September 2012

Accepted February 2013

 

Tan, C., & Liu, K., & Sun, L. (2013). A design of evaluation method for SaaS in cloud computing. Journal of Industrial Engineering and Management, 6(1), 50-72. http://dx.doi.org/10.3926/jiem.661

---------------------

Abstract:

Purpose: This paper aims to design an evaluation method that enables an organization to assess its current IT landscape and provide readiness assessment prior to Software as a Service (SaaS) adoption.  

Design/methodology/approach: The research employs a mixed of quantitative and qualitative approaches for conducting an IT application assessment. Quantitative data such as end user’s feedback on the IT applications contribute to the technical impact on efficiency and productivity. Qualitative data such as business domain, business services and IT application cost drivers are used to determine the business value of the IT applications in an organization.  

Findings: The assessment of IT applications leads to decisions on suitability of each IT application that can be migrated to cloud environment.  

Research limitations/implications: The evaluation of how a particular IT application impacts on a business service is done based on the logical interpretation. Data mining method is suggested in order to derive the patterns of the IT application capabilities.

Practical implications: This method has been applied in a local council in UK. This helps the council to decide the future status of the IT applications for cost saving purpose.

Originality/value: The proposed method has gauge various factors including the organizational, business, IT, SaaS assessment, risk assessment prior to SaaS adoption. SaaS providers and consumers hence can have a better understanding on the SaaS adoption from the top level.

Keywords: Cloud Computing, Software as a Service, Application Portfolio Management, Application Rationalization, SaaS Risk Assessment

---------------------

1. Introduction

IT cost saving has always been a major issue within organizations. Report shows that in most organizations, more than 75% of the annual IT budget is spent on the cost for operating and managing applications (Oracle, 2009). However, it is vague to identify whether or not those applications are really deriving business values to the organization. This leads to the IT application redundancy issue where similar IT applications produce similar functionalities in business. One way to disentangle this issue is through application rationalization. Gartner research shows that CIOs are finding an average of 20% immediate cost savings within 12 months of implementation along with improved IT value positioning through the combination of application rationalization effort (Oracle, 2009). Software as a Service (SaaS), one of the cloud computing service models, is one of the potential options for adding to the cost saving initiative, when the rationalized applications are migrated to the cloud.

SaaS is delivered over the internet without consumers needing to physically install the software in their infrastructure. It has been estimated that the market for SaaS is growing at 50% per year (Choudhary, 2007). There are 70% of the companies are currently using cloud based services and plans to adopt more SaaS services by moving additional application to the cloud where IT cost saving as the key motivation (Mimecast, 2010). The savings are realized through the omission of any upfront investment in software license and infrastructure costs as the charges are incurred on a pay per usage basis. In addition, SaaS has shorter implementation time of SaaS and lower failure risk compared to the conventional enterprise software implementation. However, despite the cost benefits which are attractive for SaaS consumers, there are still concerns on deploying SaaS. These concerns are meeting technical requirements, security, ease of integration and functionality (Gartner, 2009).

This research aims to design a decision making methodology for adopting SaaS. The methodology first assesses the existing IT applications, follows by analyzing the impact of SaaS implementation to the overall IT application landscape of the organization by establishing the SaaS assessment which involves risk analysis prior to adoption. This contribution of this research is in developing a mechanism for SaaS implementation that aligns with the business and IT strategy of an organization where IT cost saving as a core objective.

The paper is structured as follows: Section 2 discusses the related work in IT applications rationalization and risk analysis prior to SaaS implementation. Section 3 justifies the research design. Section 4 demonstrates the evaluation method with its set of techniques in practice through a local council in the UK. Section 5 discusses the critical evaluation of the proposed methods and section 6 draws a conclusion for this research.

2. Related Work

The Centre for the Protection of National Infrastructure describes cloud computing as a scalable package of IT services which are usually provided by a third party that owns the infrastructure (Deloitte, 2010). Cloud consumers access the services over a network on a leased basis. Cloud computing adoption decisions are always challenging due to various concerns such as cost, confidentiality and control (Khajeh-Hosseini, Greenwood, Smith, & Sommerville, 2012). Therefore, it is vital to conduct SaaS assessment prior to adoption. The assessment should incorporate factor such as strategy, IT application evaluation, IT application selection which determines the IT applications to move to the cloud, risk analysis which includes cost benefit analysis, risk encounter and mitigation solution and post deployment management.

2.1. Cloud Computing and Software as a Service

The US National Institute of Standards and Technology (NIST) defines cloud computing with five attributes: on demand self-service, ubiquitous network access, location independent resource pooling, rapid elasticity and pay per use (NIST, 2009). Cloud computing is typically divided into three service offerings and four service delivery models (Deloitte, 2010). The three service offerings are Software as a Service (SaaS), Platform as a Service (PaaS) and Infrastructure as a Service. The four cloud service delivery models are public cloud, private cloud, hybrid cloud and community cloud.

SaaS is a model of software deployment that allows on-demand licensing of software services (Carraro & Chong, 2006; Deloitte, 2010; Endrei, Ang, Arsanjani, Chua, Comte, Krogdahl et al., 2004; Laplante, Zhang & Voas, 2008). Software is typically deployed as a hosted service and accessed over the internet. SaaS has several attributes which include accessibility, reliability, configurability, scalability, costs and standardized IT based capability. From the accessibility perspective, SaaS enables consistent and frequent access when required due to critical use of software services for end users of supported business operations. The software services can be accessed through a web browser and an active internet connection. From the reliability perspective, as the software services are running from the SaaS providers’ premises, they are responsible for the security and monitoring. For example, in order to avoid any data loss, SaaS providers’ servers will backup customer data. From a configurability perspective, SaaS consumers can tailor the software service to meet their individual needs. This is a result of multi tenancy features of SaaS which allow multiple SaaS consumer configurations to be deployed on the same software service. It is an arrangement by which an organization is a tenant, SaaS provider’s servers are partitioned so that each organization works with a customized virtual application instance (Miller, 2008; Velte, Velte, & Elsenpeter, 2009). The implementation time of SaaS is shorter and failure risk is lower compared to the conventional enterprise software implementation. From the view of scalability, SaaS enables computing resources to be dynamically added or removed to match consumer needs. From the cost perspective, as SaaS is a pay per usage service, there is no upfront investment in servers and software licensing cost, leading to IT cost savings. From the standardized IT based capability perspective, SaaS is able to provide the same quality of service as existing on-site software vendors, such as timely deployment of critical patches.

Gartner survey shows the most popular application categories for SaaS deployment are messaging (email) applications, customer relationship management (CRM) applications, project management applications, financial and accounting applications, sales applications, expense management applications, HR applications which include recruitment, employee portal, travelling booking system, talent management and employee performance management applications, supply chain management applications, logistic management applications, warehouse management applications, purchasing, sourcing and e-procurement applications, office management such as web conferencing applications, training applications such as e-learning portal and company web portal (Mertz, Eschinger, Pang, & Dharmasthira, 2008). British Gas Services, the leading domestic central heating and gas appliance installation company in the UK, has adopted SaaS CRM and claims that the organization is benefiting from the SaaS solution (Saran, 2008). They have achieved cost savings as there is no upfront infrastructure cost, overall license cost is lower on a ‘pay per subscription’ basis, and the IS team is minimized as the solution is managed by the vendor. The solution only took four months to deploy instead of six to twelve months for a traditional solution.

2.2. Application Portfolio Management and Application Rationalization

Application portfolio management (APM) (Oracle, 2009) is a practice that enables organizations to manage their IT applications portfolio through application rationalization. This practice helps an organization to perform cost saving strategy through improving the IT applications performance efficiency. Application rationalization is the process to select IT applications based on business needs and prioritizing its implementation actions, to manage the value of both existing and proposed applications effectively, and to monitor priorities change and application value through continuous reviews and process adjustments in real time (Oracle, 2009). These processes are suggested to be performed, i.e. 1) application inventory, 2) application evaluation, and 3) transformation for recommendations.

Framework for Application Destiny Destination (FADD), an application rationalization framework, suggests that IT applications should be evaluated based on four principles (Kellerman & Löfgren, 2008). The four principles are business value, functional value, system quality and cost. The framework explains four ways (i.e. remove, remain, redevelop and replace) of dealing with the evaluated IT applications. FADD is a three-step process. Results are decided by a set of questionnaires. Step one decides whether an IT application should be remained or removed after assessing the business value of the organization. Step two indicates whether the IT application should be remained or changed after accessing the functional values and system quality of the IT applications. The last step questions the cost of the IT application to finally determine the changes towards the existing IT applications for either to be redeveloped or replaced. FADD offers a simple procedure to provide a preliminary skimming process of the existing IT applications in the organization. However, the mechanism of the framework is too simplistic and does not address some of the critical aspects in valuating and making decisions for the IT applications such as the risks analysis of implementing changes and business services analysis.

In comparison with FADD,  Business-Consulting Framework in Figure 1 provides a set of techniques for conducting consulting analysis (Liu, Sun, Jambari, Michell & Chong, 2011). These techniques are devised to optimize the business alignment of information systems and IT infrastructure in an organization that leads to improve productivity while minimizing cost. It is developed and adapted within Cap Gemini, using principles derived from The Open Group Architecture Framework (TOGAF), and benefiting from Problem Articulation Method (PAM). PAM is one of the methods for eliciting, analyzing and specifying user requirements with organizational semiotics as a foundation (Stamper, 1994). Organizational semiotics is the study of organization using concepts and methods of semiotics (Liu, 2000). The study is based on the fundamental observations that all organized behavior is affected through communications and interpretation of signs by people.

Figure 1. Business-Technology Alignment Consulting Framework (Liu, et al., 2011)

2.3. Factors to consider prior to SaaS Adoption

There are various types of stakeholder at different levels of abstraction in cloud computing (Anderson & Young, 2010). In the context of SaaS, there are three types of stakeholder identified: SaaS infrastructure provider or known as cloud infrastructure provider, SaaS provider and SaaS consumer. SaaS infrastructure provider owns and manages cloud computing resources which include hardware, network and system software. Market players include Amazon, IBM and Oracle offer infrastructure support which include storage, processors and virtual servers. SaaS providers serve the front end SaaS consumers by offering services such as on-demand computing, high volume computation, utility computing, data processing and software services. They usually use software running on hardware-software resources managed by a SaaS infrastructure provider. It is possible that the SaaS infrastructure provider and SaaS provider are the same vendor. SaaS consumer is classified as the user who uses on-demand computing, software services as well as utility services on the ‘pay per use’ basis. They usually use a web browser or other user interface software to access SaaS services via the internet.

Surveys, from both Mimecast and ENISA, regarding cloud computing, show the major concerns with SaaS adoption are security, loss of data control, data protection and compliance with government regulations (ENISA, 2009b; Mimecast, 2010). ENISA published a risk assessment security report to address these issues (ENISA, 2009a). It is suggested that SaaS consumers should consider having consumer-managed security controls such as encryption and identity management (Edwards, 2008). Details such as right to audit, use of physical security, protective monitoring, data segregation controls and vulnerability management processes should be included in the contract in order to secure their data in the cloud environment. In addition, SaaS consumers should consider the legislations which govern the interception and disclosure of their data for all jurisdictions in which their data are stored and transmitted across. SaaS consumers can consider pursuing a program of assurance activities on their SaaS provider to ensure contractually agreed standards are met. Showing in table 1 are the aspects a SaaS consumer should consider before adopting a SaaS service.

Table 1. SaaS aspects to consider before adoption (adapted from Deloitte, 2010; Edwards, 2008; Herbert, 2010; Wainewright, 2009)

Anderson and Young (2010) suggest that the SaaS provider should assist the SaaS consumer in understanding the benefits and risks of adopting a SaaS model. It is vital that the SaaS provider gives detailed service breakdowns for realizing the SaaS consumer’s business targets and identifies the shared responsibilities of managing their application portfolios. The SaaS provider should address issues such as how to help the SaaS consumer in controlling costs and application portfolios as well as managing changes in roles, responsibilities and organizational structure in their service proposal in order to assist the SaaS consumer to adopt and adapt to the SaaS delivery model. The SaaS provider should emphasize a life cycle approach for delivering SaaS services which support design, build, test, run, and optimization for the SaaS consumer to demonstrate their ability to support the SaaS consumer long term.

2.4. SaaS Risk Analysis and Mitigation

Table 2. SaaS aspects to consider before adoption (adapted from Brodkin, 2008; Deloitte, 2010; ENISA, 2009a; Finch, 2008; Spinola, 2009)

The purpose of conducting risk analysis is to implement necessary controls in SaaS implementation and to document management due diligence to ensure that sufficient support is provided for an organization and to meet their objectives (Peltier, 2004). The main deliverables of risk analysis are identifying threats, determining the probability of threat occurrence, establishing controls so that the risk can be reduced to an acceptable level. A cost benefit analysis is suggested after the controls are identified in order to determine the resources allocated and cost impact of their implementation. The cost benefit analysis shall include cost of SaaS implementation, allocating resources in supporting the SaaS service after implementation and to provide training to end user in using the newly implemented SaaS service. On the other hand, risk mitigation is a systematic methodology in order to reduce risk (Peltier, 2004). The common methods are risk assumption, risk alleviation, risk avoidance, risk limitation, risk planning and risk transference. Showing in Table 2 is the summary of major risks and mitigation methods for SaaS implementation.

3. Research Methodology

The aim of this project is to remove redundant IT applications by looking into the SaaS option in order to align with the council’s IT cost saving strategy. In order to ensure that the SaaS adoption is bringing business value to an organization, the top down approach is applied in this research and can be summarized in four stages. In stage one, the council’s current IT landscape is examined in order to grasp the views on how IT applications support the business process. This includes the study on the organization’s business goals, business strategies and the value of business services and IT applications. In stage two, the application rationalization process will be conducted. The aim of this process is to select the applications based on business needs and help the decision makers to make appropriate adjustments. In stage three SaaS readiness will be evaluated, this involves a risk assessment of potential applications that can be moved to the cloud and the cost impact of the transition. This is to ensure that the transition achieves the cost saving objectives. In stage four, a roadmap for SaaS will be derived. On the nature of the research work, a mixed quantitative and qualitative approaches has been adopted  in the design of the research (Creswell & Clark, 2007). A data source was chosen for this ‘Application Rationalization’ project from a local UK council.

Data were collected in four main areas which are business domain at organizational level, such as business and IT strategy, business services information, IT application information including the IT infrastructure, supplier and cost and information services. These data are used for measuring the business value of IT applications. A questionnaire was implemented in order to gather the users’ feedback on their perceived value of the IT application in supporting their day-to-day jobs in the council. The users are required to rate each IT application’s attributes on a scale of 0-5 in the questionnaire. The results were computed based on the assessment aspects in Table 3 and rating criteria in Table 4.

Table 3. Attributes description

Table 4. Rating criteria description

The qualitative data from written or oral formulations were collected through in depth interview are conducted in order to collect business domain, business services and IT application cost information. Such technique allowed the project to enrich data being collected from both individuals and groups (Easterby-Smith, Thorpe, & Lowe, 2002). Ten IT managers were interviewed for this purpose. A few finance staffs were also interviewed in order to summon the financial data of the IT applications. This is intended to gather the annual costs associated with each application used by the council. Figure 2 shows the IT application cost break down. The cost data collected will then be analyzed and used to help in the process of suggesting changes to the current application portfolio. 

Figure 2. Application Financial Questionnaire

The collected information on business domain, business services and IT applications serves as the input for this research. The collected data are then simulated in the case tool as illustrated in section 4.1 and eventually recommends which IT application that could opt in the SaaS option. There are two stages of simulation. The first stage of simulation determines if changes should be made to an IT application by comparing the business criticality (BC) and IT application value (ITAV). This is to ensure that a particular IT application is evaluated based on its contributions to a business service.

Table 5. SaaS Assessment attribute’s description

At the same time, this helps to identify the number of IT applications that are functionally performing the same thing. Once the status of the IT applications is determined, the second simulation will determine the IT applications that could opt in to SaaS based on the predefined SaaS categories by Gartner (2008). The result from these two simulations serves as a preliminary analysis prior to the SaaS assessment. SaaS assessment that includes SaaS risk assessment and cost comparison between the deployment cost and implementation cost will then determine whether the IT application can be replaced by SaaS. Each application is evaluated using attributes as shown in table 5. Ten IT managers have been interviewed for this purpose. The method returns 1 for Yes / Pass and 0 for No / Fail. Eventually, if the particular IT application is ready to be deployed as SaaS solution, the Ready to Go column will return 1 and 0 if it is not. Eventually a SaaS adoption summary will be provided. The final decision is made by the council’s higher management.

4. The Evaluation Method for SaaS Adoption

Figure 4. The evaluation method architecture

Figure 4 presents the evaluation method for SaaS adoption. This method consists of a set of structural analysis techniques that aids the decision making process for SaaS adoption. This method enables the analysis to establish a holistic view of the organization’s current IT landscape together with SaaS assessment factors prior to SaaS adoption. Therefore, it helps the potential SaaS consumers to identify the potential IT applications that could be replaced with SaaS applications by considering the business factors at the same time. On the other hand, the SaaS providers can identify the key factors that they should consider before delivering SaaS services to their consumers.

 

4.1. The Evaluation Method and Assessment Process

The evaluation method comprises of four stages. Stage one: Business Service Architecture Analysis focuses on representing the knowledge of the organization’s business service foundation. This stage comprises of six analysis components, Business Domain Analysis, Stakeholder Analysis, Business Services Portfolio, Business Services Valuation, IT Application Portfolio and IT Application Valuation. The valuation for both business services and IT application is in a form of questionnaires where the valuation criterion is assigned with a weight, which is defined by the agreements between the consultants and the organizations decision makers. Stakeholders are then required to rate each criteria using a scale of 1 to 5. The description of the rating is as shown in Table 4. The valuation implements threshold values to serve as benchmarking to determine the overall value of the business service and IT application. The threshold value is usually determined by the business unit head or the project sponsor. The computation of the values together with the result interpretation for both business services and IT application is shown in Table 6.

Table 6. Business services and IT application valuation result interpretation

Stage Two: IS Services Analysis focus on application rationalization by evaluating the IS services performance derived in the previous stage and eventually decide the potential IT applications that could be replaced with SaaS solution and would deliver business value to the organization. This stage has two analysis components, IS services (First Evaluation Analysis) and IS services (Second Evaluation Analysis). IS Services (First Evaluation Analysis) is the component that evaluates the criticality of the IT application in the relationship between the business services and IT application portfolio. The evaluation result is based on a logical interpretation by comparing the business criticality (BC) and IT application value (ITAV). If multiple priorities exist within a combination then the highest priority value will be returned as the result.

IS Services (Second Evaluation Analysis) is the component that extends the analysis on the IT applications from the outcome of IS Services (First Evaluation Analysis). This analysis populates IT applications where the “First Evaluation” equals to “Replace / Redevelop” and “Application Type” is equal to the application categories for SaaS deployment suggested by Gartner (Gartner, 2008). Stage Three: SaaS Assessment focuses on performing SaaS readiness assessment which involves risk assessment to determine potential SaaS applications (potential applications that can be moved to the cloud). It consists of two analysis components, SaaS Assessment and SaaS Risk Assessment. Stage Four: SaaS Adoption Summary provides summaries of the complete SaaS adoption results including its cost impact for the SaaS transition. It delivers the final outcome from all the analysis and provides the recommendation for the IT application that should be remain or replaced by considering the stakeholder’s decision.

The evaluation method ensures practitioners who can easily understand the analysis of each of the IT component and the relationship between them. It is supported by the business services portfolio in which all business services and their criticality to the business of council operations. Besides, total cost of ownership (TCO) element, the business services and IT application valuation adopted the standard rating scale, descriptions and result interpretation. This enables consistent results when mapping business services and IT application value. A threshold element has been added to allow organizations to prioritize services based upon their own criteria.

4.2. The Application of the Evaluation Method

The council is looking for ways to save £945,000 per annum from their IT budget. The challenges faced by the council have been observed and the collected data are applied to stage one and two of this method. Stage three and four are validated via the simulation of the analysis result from previous stages, and the assumptions made based on literature review. The observations and preliminary analysis of the council have been identified: 1) there is no proper platform to capture information and status of existing council applications 2) there is no clear information on IT spending and 3) there are redundant applications serving the same purposes. Therefore, it is vital to have an application inventory in order to manage the IT applications effectively. This ensures that changes made to applications are aligned with the organization’s IT and business strategy. The key results of the implementation of the evaluation method towards providing recommendations for IT applications to adopt SaaS are presented in the following sections.

Business Services Architecture Analysis

A business services portfolio is developed for the council. It contains information of each business service, stakeholders associated with the business service, and the IT applications that support the business service’s performance. The business service valuation is done for each business service. During the interview process, it was discovered that each business service plays an equally important role to the council in providing services to the residents. Therefore, these business services share the same threshold value. The IT Application Portfolio analysis is performed based on the information collected from the council. The analysis produces the information of the council’s IT application, business service, supplier and cost information. The analysis reveals that the council has 272 IT applications and there are redundant applications supporting the same business services. Examples for the redundancy identified are visible in the CRM service where there are four applications supporting it and in Payment service, ten applications have been identified to support it. Each IT application is then evaluated using the IT Application Valuation analysis. As a result, 48 applications returns the value ‘High’, two applications returns the value ‘Low’ and 222 applications returns the value ‘Medium’ level of support to the business service it supports.

IS Services Architecture Analysis

By comparing the business criticality and IT application values performed in IS Services (First Evaluation Analysis), there are 70 applications fall into the ‘Replace / Redevelop’ category. The IS Services (Second Evaluation Analysis) further results 20 applications in five SaaS application categories: 1) CRM (1 application), 2) HR (3 applications), 3) Messaging (1 application), 4) Web portals (3 applications) and 5) Office (12 applications). This indicates that these applications are potential SaaS applications. Figure 5 presents the sample result of IS Services (Second Evaluation Analysis).

Figure 5. IS Services (Second Evaluation Analysis)

SaaS Assessment and SaaS Risk Assessment

SaaS Assessment analysis is then performed to the 20 applications for its accessibility, functionality, reliability, configurability, scalability, availability, standardize IT based capability, service delivery management, SLA and status visibility. SaaS Risk Assessment is performed to the applications for its performance towards the following risks: data lock in, application lock in, data confidentiality & integrity, data disposal, data viability, investigative support, privileged user access, data location, continuity management, performance management, regulation & legislation and change management. The result suggests that the identified threats could critically impact the council particularly when dealing with confidential information. The final outcomes from both analyses have produced the SaaS readiness value for each IT applications. The result returns ‘1’ for ‘Pass’ and ‘0’ for ‘Fail’. The logic to derive the SaaS readiness is as follow.

The ‘SaaS Readiness’ returns:

 ‘1’ if total number of ‘1’ > ‘0’

 ‘0’ if total number of ‘1’ < ‘0’

Another factor that is crucial for assessing the IT applications for SaaS adoption is the cost factor. The analysis requires two cost values for each IT applications, the current application cost and the estimated SaaS deployment cost. The current application cost is collected from the IT inventory and the estimated SaaS deployment cost is provided by the SaaS provider. Comparison between the two costs is then performed using the logic presented as follows.

‘1’ if SaaS deployment cost < Current application cost

‘0’ if SaaS deployment cost > Current application cost

The SaaS readiness and Cost Comparison result values are then populated into the SaaS Adoption Summary Analysis for the final analysis. Figure 6 shows the sample result of SaaS Assessment Analysis.

SaaS Adoption Summary

This analysis produces a summary that contains recommendations for future status of the IT applications (remain/replaced). The recommendations are derived by mapping the SaaS readiness and cost comparison results produced in the previous analysis. The logic for the mapping is as follows.

If value of “SaaS readiness” and “Cost comparison” is ‘1’,

then ‘Recommendation’ returns ‘Replace’,

else ‘Recommendation’ returns ‘Remain’.

Figure 6. The SaaS assessment analysis

In summary, this method suggests that 8 applications are recommended to be “replaced” with SaaS based applications and 12 applications will “remain” as-is. Those “remain” applications are unlikely to be replaced due to its compliance with the council’s IT policy.

5. Discussion

The derivation of the attributes including the risk factors in this context is generalized in order to meet the basics requirements for SaaS implementation for an organization. SaaS is one of the cloud computing offerings. This method has the capacity to be extended to include the assessment of other components of cloud computing.

IT applications are evaluated based upon the combination of the business criticality (BC) and IT application value (ITAV) results. As there are a many to many relationship between stakeholders, business services, IT applications, each application may have different ratings for different business services by different stakeholder. In order to derive a definitive result, the average value across IT application evaluation will be computed. Thresholds as shown in table 6 are implemented for both business services and IT application valuation as different organization may have different perspectives when defining business service and IT application value. This method is designed using optional attributes, allowing organizations to adapt the model to meet their own definition of ratings. The use of binary data enables automation of the decision tree concept. Results can be calculated to support the SaaS adoption decision making process by taking into consideration applications, costs and risk factors.

This method extracts the intermediate assessment results from each stage, and transforms the result to integrate with further stages in the robust manner. With the introduction of business services and IT applications concept, the assessment provided the stakeholders with the in-depth understanding of the business domain and then setting the sensible threshold values. The current relationship between the business criticality and IT application valuation is based on the logical interpretation; this could be enhanced by using data mining methods. In order to see which relationship model fits the organization’s requirements the most scenarios can be developed that use different threshold values. This method involves weighting the stakeholder analysis, where each stakeholder is tagged with a numerical weight and business services valuation, and each business service evaluation criterion is tagged with a numerical weight. Currently the weighting is decided semantically by either the consultants or stakeholders. Therefore, it is suggested that, for example the stakeholder weighting can gauge with a stakeholder power impact analysis.

This research focuses on the SaaS adoption, which is one of the cloud computing service offering. It is suggested that the technical component should be assessed, especially when dealing with applications that have multiple interfaces with other applications. This method can be enhanced as a cloud computing assessment that assesses which is the best cloud offerings to be adopted. It could be SaaS, PaaS, IaaS, or combination of the service offerings.

In addition, this method also serves as an IT application assessment tool which assists the IT managers to make decisions on SaaS adoption in the council. However, there are some limitations of the evaluation method. For example, this method has a dependency on the cost factor as it is one a core attributes to decide whether the adoption of SaaS should happen. The validation process shows that there could be a challenge in getting the financial information of the IT applications. Therefore, as an alternative, an engineering approach should be derived in order to serve the decision making purpose. It is suggested that a layer of weighting is introduced to this method, where all entities are associated with a weighting. For example, if the financial information cannot be obtained, the cost attribute will have the lowest weight, and other attributes will have higher weightings, thus giving them more influence in the decision making process. On the other hand, this method has currently provided entities with a generic set of attributes. It is suggested that industry templates of predefined attributes could be developed in the future.

6. Conclusion

In summary, this research covers the major concerns organizations have when adopting cloud computing. The focus of this research is on one of the cloud computing offerings, Software as a Service (SaaS). This is conducted by developing a framework using a top down approach. This starts by assessing the IT landscape in an organization, followed by integrating a risk assessment and finally making recommendations on which applications can be moved to the cloud.

Both SaaS consumers and SaaS providers can benefit from this method. This method provides SaaS consumers a clear overview and consideration factors that can be evaluated before adopting SaaS. Consideration factors include organization, business, IT application, SaaS assessment and cost. The SaaS assessment has concluded concerns and risk factor. The result has shown that this method is capable of analyzing IT applications based on their criticality to the business services and their suitability for transition to the cloud at the fundamental level. As cost is one of the key motivations for SaaS adoption, this method gives a clear comparison of the current application cost and the cost of SaaS implementation which includes deployment and risk mitigation costs. In addition, SaaS consumers may consider establishing a long term contract with the SaaS provider in order to negotiate lower prices for future SaaS implementations and maintenance.

This method also serves as a fundamental requirement analysis for SaaS providers by providing a clear view on what factors consumers consider before adopting SaaS. SaaS providers can consider implementing those factors into their SaaS service package. In addition, the strategic elements such as implementations strategy, SaaS migration strategy and post implementation strategy can add value to their service package. SaaS providers may consider working with their strategic partners to share knowledge and skills in order to provide a better service to consumers.

Acknowledgements

Dr Sam Chong, former Sector CTO at Capgemini provided valuable guidance on the initial formation of the work. Capgemini UK facilitated the case study with the local council in the UK.

References

Anderson, D., & Young, A. (2010). Market Trends: Application Services, Worldwide, 2010; Will Providers Deliver What Clients Really Want?. Gartner.

Brodkin, J. (2008). Gartner: Seven cloud-computing security risks, from http://www.infoworld.com/d/security-central/gartner-seven-cloud-computing-security-risks-853?page=0,1

Carraro, G., & Chong, F. (2006). Software as a service (SaaS): An enterprise perspective. MSDN Solution Architecture Center.

Choudhary, V. (2007). Software as a service: Implications for investment in software development. Proceedings of 40th Hawaii International Conference on System Sciences -2007.

Creswell, J.W., & Clark, V.L.P. (2007). Designing and conducting mixed methods research: Wiley Online Library.

Deloitte. (2010). Information Security Briefing 01/2010 Cloud Computing: On behalf of CPNI by Deloitte.

Easterby-Smith, M., Thorpe, R., & Lowe, A. (2002). Management research: An introduction: Sage Publications Ltd.

Edwards, C. (2008). Legal Download 2.0 – Advantages and Risks of Software-as –a-service  Retrieved 16/02/2010, from http://www.itp.net/518351-legal-download-20-advantages-and-risks-of-software-as-a-service

Endrei, M., Ang, J., Arsanjani, A., Chua, S., Comte, P., Krogdahl, P., et al. (2004). Patterns: service-oriented architecture and web services: IBM Corporation, International Technical Support Organization.

ENISA. (2009a). Cloud Computing: Benefits, risks and recommendations for information security. European Network and Information Security Agency (ENISA).

ENISA. (2009b). Survey: An SME perspective on Cloud Computing. European Network and Information Security Agency (ENISA).

Finch, A. (2008). The Rise of SaaS and Your Regulatory Risks, from http://www.ecommercetimes.com/story/61448.html?wlc=1268221100&wlc=1291074561

Gartner. (2008). User Survey Analysis: Software as a Service, Enterprise Application Markets, Worldwide. Gartner, Inc.

Gartner. (2009). Gartner Survey Shows Many Users are Underwhelmed by Their Experiences of SaaS.  Retrieved 15/2/2010, from http://www.gartner.com/it/page.jsp?id=1062512

Herbert, L. (2010). How to Get Strategic About SaaS: 5 Key Considerations, from http://www.cio.com/article/639015/How_to_Get_Strategic_About_SaaS_5_Key_Considerations?page=1&taxonomyId=3024

Kellerman, J., & Löfgren, P. (2008). Application Portfolio Management. rapport nr.: Report/IT University of Göteborg 2008: 055.

Khajeh-Hosseini, A., Greenwood, D., Smith, J.W., & Sommerville, I. (2012). The cloud adoption toolkit: supporting cloud adoption decisions in the enterprise. Software: Practice and Experience, 42(4), 447-465.  http://dx.doi.org/10.1002/spe.1072

Laplante, P.A., Zhang, J., & Voas, J. (2008). What's in a Name? Distinguishing between SaaS and SOA. It Professional, 10(3), 46-50. http://dx.doi.org/10.1109/MITP.2008.60

Liu, K. (2000). Semiotics in information systems engineering: Cambridge Univ Pr.

Liu, K., Sun, L., Jambari, D., Michell, V., & Chong, S. (2011). A design of business-technology alignment consulting framework. Proceedings of the Advanced Information Systems Engineering. http://dx.doi.org/10.1007/978-3-642-21640-4_32

Mertz, S., Eschinger, C., Pang, C., & Dharmasthira, Y. (2008). User Survey Analysis: Software as a Service, Enterprise Application Markets, Worldwide, 2008: Gartner.

Miller, M. (2008). Cloud computing: Web-based applications that change the way you work and collaborate online. Que publishing.

Mimecast. (2010). Cloud Computing Adoption Survey. Mimecast, UK.

NIST. (2009). NIST Definition of Cloud Computing v15  Retrieved 16/02/2010, from http://csrc.nist.gov/groups/SNS/cloud-computing/index.html

Oracle. (2009). Benefits of Application Rationalization: Reduce Costs and Improve Service with a Systematic Approach. Oracle Corporation, CA.

Peltier, T.R. (2004). Risk analysis and risk management. EDPACS, 32(3), 1-17. http://dx.doi.org/10.1201/1079/44581.32.3.20040901/83426.1

Saran, C. (2008). Case studies: SaaS in action  Retrieved 16/02/2010, from http://www.computerweekly.com/Articles/2008/09/11/231932/Case-studies-SaaS-in-action.htm

Spinola, M. (2009). An Essential Guide to Possibilities and Risks of Cloud Computing: a Pragmatic Effective and Hype Free Approach for Strategic Enterprise Decision Making. Retrieved 15/04/2011.

Stamper, R. (1994). Social norms in requirements analysis: an outline of MEASUR.

Velte, T., Velte, A., & Elsenpeter, R. (2009). Cloud Computing, A Practical Approach: McGraw-Hill, Inc.

Wainewright, P. (2009). What to look for in a SaaS Vendor  Retrieved 16/02/2009, from http://blogs.zdnet.com/SAAS/?p=200