Procurement and implementation of Information Systems is especially challenging due to the complexity of procuring unknown technology and the importance an information system has for different stakeholders in an organization. Procurement and implementation of information systems (IS) and services provides several challenges and problems to the stakeholders involved in the procurement and their implementation processes. However, these are not well established or understood, and there is a knowledge gap that needs to be covered. This paper presents results from a Delphi study, which involved 46 experienced IT managers, HRIS officers, and vendor representatives from various industries. The participants identified 98 challenges and problems related to IS procurement and implementation, and subsequently ranked the relative importance of the top critical issues. The study supports findings from previous research related to diverging stakeholder goals; challenges in balancing between objectives; in requirement specifications; and in too narrow cost focus. In addition to providing empirical confirmation of these previous propositions the study revealed new findings, such as benefits realization in IS procurement and implementation; critical issues of technological integration, negative impacts of IS constraints on stakeholders and measures to be taken to resolve the negative impacts of IS procurement and implementation and losses occurred due to IS constraints. The results provide a rich overview of IS procurement and implementation challenges and its negative impact on stakeholders in corporate sectors.
Procurement and implementation of information systems (IS) and related services is challenging compared to acquisition of more standardized goods and services. Information systems often need to be customized to the needs of the industrial units (Keiichiro & Hajime, 2005). Procurement decisions are made early in the procurement process, when requirements are still uncertain (Saarinen & Vepsäläinen, 1994). The buyer may have to compare between competing, complex system options. Information systems can support this process (Davila et al., 2003), but research shows difficulties in implementing e-procurement. Furthermore e-procurement offers limited support in the process focusing mainly on the selection of vendors, but less on other parts of the process such as requirements specification, negotiations and contract monitoring.
This research focuses on information systems that are implemented for specific organizational purposes, such as enterprise resource planning systems and e-services tailored for the buyer’s needs and various negative impacts reflected due to IT constraints on stakeholders. We thus exclude acquisition of off-the-shelf software from this study. Outsourcing of IS development and implementation is a relevant critical issue in this research context, as complex systems often require customization and involve contracting with a vendor to tailor an existing information system or develop a new system altogether.
The IS procurement and implementation process is in itself challenging, as is the complexity of procuring new or unknown technology and then making every one familiar to work with the same IS. However, these challenges are not well researched and this project seeks to fill this gap.
The research question concerns the challenges and dilemmas that are typically faced in procurement and implementation of information systems and related services in the corporate sector and how it impacts stakeholders of the organisation.
IS procurement and implementation of Information Systems has gained little attention from researchers, but some of the previous work on it in general carries relevance for this research question. This is introduced below. A summary of challenges and negative impacts specific to Information Systems constraints on stakeholders, and provide an overview of the previous literature in table 1.
IS procurement and Implementation constraints and its negative impacts
One of the previously identified challenges and negative impacts concerns the critical issue of various internal stakeholders with conflicting goals. Organizational buying involves multiple participants in a process (Wind & Thomas, 2001) where many purchasing decisions are influenced by various members of the buying center (Spekman & Stern, 1979). In addition, an organisation involves the complexity of satisfying needs of different stakeholders.
One line of previous work has focused on the negative impacts of conflicting goals in IS procurement and implementation constraints-
· Balancing the dynamic tension between
a) Competing socioeconomic objectives and
b) National economic interests and global competition as required by regional and international trade agreements;
· Satisfying the requirements of fairness, equity, and transparency; and
· Maintaining an overarching focus on maximizing competition.
There is also the critical issue of information asymmetry when procuring and implementing services from IS consultants (Dawson et al., 2011). Agency theory suggests contracts and monitoring of the work to limit opportunism from a vendor, but this may not be sufficient to cope with the problem as consultants have more knowledge of the problem area than the organisation. The challenge of consultant opportunism comes in addition to the challenge of competing interests from internal stakeholders and may necessitate a complex set of strategies.
The contemporary literature on IS procurement and implementation challenges and public procurement remains largely without an established theoretical base and there is limited empirical data to validate the conceptual and normative recommendations. The literature identifies a number of potential negative impacts, but there is little systematic research on additional challenges faced by stakeholders in IS procurement and implementation. A study of 4 ERP procurements in private sector shows the importance of adopting a stakeholder approach (Poon & Yu, 2010). The challenge of different stakeholders may be more important in the public sector than in private sector, but there is limited research on how it interplays with other challenges. The main findings from previous research on challenges related to procurement & Implementation of Information Systems and their negative impacts / challenges are summed up in Table 1. The table further shows the research approach and the analytical lens for these studies.
Table 1: Summary of findings from previous research on negative impacts of IS constraints on stakeholder
Challenge / Negative Impacts on Stakeholders | Proposition / critical incidents | Type of study | Theory |
Various internal stakeholders An organisation faces a variety of stakeholders, placing demands and constraints on managers (Boyne 2002) Gaps between project goals and stakeholder goals, both internal and external (Pan 2005) | Demands and constraints from different stakeholders may be in conflict Underestimation of stake-holder groups may lead to problems in terms of resistance | Literature review Case study of development of information system (for e-procurement | Stakeholder theory |
Governance of procurement and implementation processes over time (Poon & Yu, 2010) | Adopting a stakeholder approach and preparing evaluation criteria are critical success factors | Case study of 4 ERP procurements in private companies | Micro-politics |
Information asymmetry (Dawson et al., 2011) | Consultants are difficult to control through contracts due to information asymmetry; there are more opportunities for opportunism | Interviews with 15 experienced IS consultants and procurers | Agency theory, principal-professional lens |
Challenge / negative impacts | Proposition | Type of study | Theory |
Limited interest from vendors, due to payment model and standard government contracts | Not enough competition, and the buyer may not be able to get optimal price or quality | Analysis of guidance and model contracts for UK government IT projects (Doshi, 2005) Case study of two public IS procurements (Moe et al., 2006) | No specific theory |
IS procurement balances between socio-economic objectives organisations have more ambiguous goals Local vendors and vendors representing minorities may be favored | This may create dilemmas between conflicting goals | Subsequent research supports Thais (2001) claim. Boyne (2002) finds support for ambiguous goals in literature review Bartle and Korosec (2003) find that social preferences are used by American state governments | Conflicting goals; these conflicts can be between different stakeholders |
Specifying requirements before announcing tender Information Systems may have ill-defined scope and unclear requirements | The requirement may ask for the wrong system | Case study of two public IS procurements (Moe et al., 2006) Findings from two cases indicate that partnership may be better suited to complex procurements (Lawther & Martin, 2005) | None |
Focusing on lifecycle cost and not just initial procurement costs | If managers do not adopt a long-term perspective for valuation, they may end up with higher lifecycle costs | Survey from materials procurement in Norwegian Army (Tysseland, 2008) | Agency theory, information asymmetry, project uncertainty |
We saw a need for research to identify and prioritize the challenges and negative impacts due to IS constraints and assess how they are related. Delphi method is used for this research, this can be used to develop an overview of what challenges and problems are most prominent in a field (Okoli & Pawlowski, 2004). In this context, the challenges represent factors that may have negative impact on stakeholders arising due to procurement and implementation process constraints in the resulting system.
I chose to follow the process steps recommended for ranking-type Delphi studies (Okoli & Pawlowski, 2004; Schmidt, 1997) in order to identify, select, and rank the observed problems and challenges.
The Delphi method is useful in complex, immature fields involving expert judgment (Gupta & Clarke, 1996; Rowe & Wright, 1999). It fits especially well in situations where the experts are geographically scattered. The method formalizes communications between researchers and experts in order to extract unbiased information based on the experts’ opinions. The key features that characterize the Delphi method are anonymity, multiple iterations, controlled feedback, and statistical aggregation of the group response (Rowe & Wright, 1999). Potential disadvantages include lengthy process, potential researcher influence on responses based on formulation of the questions, and difficulties due to the fact that the experts never meet in person (Murry & Hammons, 1995).
Composition of the expert panels
First, I selected the experts for the study. It was limited to inviting practitioners only, from different types of organizations of a reasonable size (hospitals, banks, PSUs). I also selected experts from vendors who provide Information systems and services and have a considerable portion of this market. This design involves three expert panels: IT managers, HRIS officers, and vendor representatives
I contacted the experts, whom I knew from previous projects or through professional networks by e-mail and phone, inviting them to participate and explaining the purpose and process of my research. I further asked them to nominate other experts who satisfied my selection criteria.
The IT manager panel included 18 participants, the HRIS officer panel 17 participants, and the vendor panel 11 participants.
Data collection process is divided into three phases: -
brainstorming, narrowing-down, and ranking, as recommended by Schmidt (1997) and Okoli & Pawlowski (2004).
In the first phase I brainstormed critical issues related to the research question. A welcome letter was sent to the participants by e-mail. Each expert was asked to list at least six challenges for or dilemmas of IS procurement and implementation. I asked them to give each challenge a name, a definition, the causes for each challenge and the consequences that would occur if they were not managed. By answering this, the experts gave a structured explanation of each challenge. For example, one challenge is writing clear requirements specifications. One HRISO explained that this was due to the strict requirements for tender format and the low threshold for official complaints. This could lead to the vendors taking advantages of shortcomings in the specification, and to the procuring entity ending up with making the wrong choices. One of the procurement managers explained that the challenge was caused by a lack of holistic understanding of the business processes, and this could lead to a lot of change orders and to procurement of modules that are not implemented.
The experts e-mailed their lists to me and After collecting the replies, I combined the critical issues into a single list, removed exact duplicates, and unified terminology. I collated the responses independently, before comparing and consolidating the individually constructed lists. I sent my consolidated list of 96 challenges back to the experts to ensure I had not eliminated any challenges in this phase and that I had not misinterpreted any critical issues. This step resulted in the addition of two more items. The entire consolidated list of 98 challenges and dilemmas from the brainstorming is presented in Appendix A.
In the second phase I narrowed the list down to a manageable number of the most important critical issues. In each panel, each expert defined around 20 critical issues that they considered the most important. The presentation order of the full list of critical issues was randomized to avoid bias in selection of the most important challenges, based on a factor’s sequence in the list.
This phase resulted in a list of 19 critical issues, which were selected as follows. First, I selected a “top ten” list based on the votes in total across the three panels. This resulted in 13 challenges in total, as the challenges ranked from 10 to 13 got the same number of votes. Then I checked whether there were large differences between the panel selections. Kendall’s tau (a measure to study ranking correlations between different panels) values showed some correlations between the panels selections for the narrowed-down lists. However, all the correlations were less than 0.5 (Table 2), and values below this threshold is a sign of two rankings not being relatively similar. So I decided to include challenges chosen by more than 50% (Schmidt, 1997) of members in each particular panel. This step assured that each panel had its challenges represented in the narrowed-down list. It resulted in six additional challenges to be included for further analysis, giving a total of 19 in the list.
In the third phase the relative importance of the top 19 critical issues were ranked. Since the Kendall’s tau values between all the pairs of the panels were below 0.5, we chose to do the ranking separately for all the three panels. By dividing the experts into three separate panels, I expected to reveal potential differences in challenges between these three stakeholder groups.
The third phase was carried out in two rounds. In Delphi studies, the number of ranking rounds should depend on whether each panel reaches either an acceptable level of consensus or a state where the level of consensus stagnates. Kendall’s coefficient of concordance (W) was used to measure the level of consensus within each of the panels.
The results from the first round of ranking were fed back to the panel members. They were asked to reflect on their ranking compared to the group’s average, and then re-rank the challenges. Kendall’s tau values on the first ranking round (Table 2) showed some interesting results. While the top critical issues from the narrowing-down phase correlated between all panels to some extent, the dissents between the vendor panel and the two other panels increased after the ranking rounds. The vendors’ selection did not correlate significantly with the two other groups. The IT managers’ and HRISOs’ rankings continued to correlate, however some factors were very differently valued by the two panels. Hence, a panel-wise discussion and comparison of the ranking results is legitimate.
Schmidt (1997) recommends a concordance level of W = 0.7 to indicate a high level of agreement among the respondents in each panel. Ideally, the ranking rounds should continue either until that level is reached, till the concordance level does not increase further between two consecutive ranking rounds, or till one more round is no longer considered feasible (Schmidt, 1997). I decided to stop ranking after two rounds, due to several indications that the panel members were not willing to participate in more rounds. I had to send several reminders on the second round, and expected to lose more panel members if I continued one more round. One representative of the vendor group had dropped out of the study between the first two rounds, and more dropouts would have weakened the reliability of yet another ranking. I had gained a moderate consensus (W > 0.5) in two of the groups (IT managers and vendors), whereas the HRISO group consensus was low (W > 0.3) to moderate (Tables 3-5). The biggest relative changes within each panel were maximally two positions up or down, so we are confident our results correctly ranks the critical issues most important to the panelists.
Table 2: Kendall’s tau values between the three panels
| IT managers *HRISOs | IT managers *Vendors | HRISOs*Vendors |
Narrowing-down phase (all 98 items) | 0.474 (sig. 0.000) | 0.205 (sig. 0.006) | 0.234 (sig. 0.004) |
Ranking round 1 (top 19 items) | 0.471 (sig. 0.002) | -0.106 (sig. 0.585) | -0.076 (sig. 0.710) |
Ranking round 2 (top 19 items) | 0.450 (sig. 0.008) | -0.112 (sig. 0.584) | -0.088 (sig. 0.681) |
The following tables present challenge rankings after the first and second round for each of the three panels. There were some minor changes in the ranking order between the first and second round, but overall the top-ranked challenges had a higher score (closer to 1), and the lower-ranked challenges had a lower score (closer to 19) in the second round.
As the results in tables 3-5 below show, the three groups ranked challenges somewhat differently. “Change of work processes and benefits realization” was ranked as the most important challenge by IT procuring officers, with an average ranking of 2.2. HRISOs ranked “Clear requirements specification” as the most important challenge, with an average ranking of 4.3. This challenge was not considered much more important than the next two challenges. “Finding and using good assessment criteria” received an average ranking of 4.5, and “Integration, compatibility” received an average ranking of 4.6 from the CIO´s. Vendor representatives differed from these two groups, ranking “Too much focus on costs” as the most important challenge with an average of 2.0.
Table 3: Ranking results: IT Procurement & Implementation managers
Challenge | Mean ranks | |||
Rank | Critical issue | Round 1 (N=18) | Round 2 (N=18) | |
1. | Change of work processes and benefits realization | 5.0 | 2.2 | |
2. | Clear requirements specification | 7.1 | 4.7 | |
3. | Integration, compatibility | 7.9 | 5.1 | |
4. | Lack of coordination and standardization | 8.1 | 6.1 | |
5. | Weighing / prioritizing the assessment criteria | 8.2 | 6.4 | |
6. | Complete requirements | 8.7 | 7.6 | |
7. | Frame agreements | 9.5 | 7.9 | |
8. | Procurement competence | 8.9 | 8.4 | |
9. | Cooperation between different stakeholders | 10.1 | 8.8 | |
10. | Tendering obligations may conflict with long-term planning | 10.1 | 10.6 | |
11. | Monopoly-resembling vendor conditions | 10.3 | 11.2 | |
12. | Too much focus on costs | 10.6 | 11.3 | |
13. | Municipal cooperation is challenging | 11.0 | 11.7 | |
14. | Finding and using suitable assessment criteria | 11.0 | 12.5 | |
15. | Partnership and innovation are hindered | 11.2 | 13.3 | |
16. | Complex regulations | 12.6 | 14.9 | |
17. | Vendors tend to oversell | 12.7 | 15.0 | |
18. | The vendors don’t get to show their qualities | 14.1 | 15.4 | |
19. | Feasible requirements | 13.0 | 15.7 | |
Kendall’s W | 0.160 | 0.537 | ||
Table 4: Ranking results: HRISOs
Challenges | Mean ranks | |||
Rank | Critical issue | Round 1 (N=17) | Round 2 (N=17) | |
1. | Clear requirements specification | 6.4 | 4.3 | |
2. | Finding and using good assessment criteria | 6.5 | 4.5 | |
3. | Integration, compatibility | 5.1 | 4.6 | |
4. | Lack of coordination and standardization | 8.2 | 7.4 | |
5. | Weighing / prioritizing the assessment criteria | 8.9 | 7.8 | |
6. | Partnership and innovation are hindered | 8.4 | 8.2 | |
7. | Change of work processes and benefits realization | 7.5 | 8.5 | |
8. | Too much focus on costs | 8.3 | 9.3 | |
9. | Tendering obligations may conflict with long-term planning | 9.8 | 9.5 | |
10. | Complex regulations | 9.8 | 9.6 | |
11. | Frame agreements | 9.3 | 9.8 | |
12. | Cooperation between different stakeholders | 10.1 | 10.4 | |
13. | Procurement competence | 11.0 | 11.1 | |
14. | Complete requirements | 12.2 | 12.2 | |
15. | Municipal cooperation is challenging | 11.2 | 12.5 | |
16. | Vendors tend to oversell | 12.8 | 13.8 | |
17. | Monopoly-resembling vendor conditions | 13.3 | 14.4 | |
18. | The vendors don’t get to show their qualities | 15.5 | 15.9 | |
19. | Feasible requirements | 15.6 | 16.2 | |
Kendall’s W | 0.268 | 0.391 | ||
Table 5: Ranking results: Vendors
Challenges | Mean ranks | |||
Rank | Critical issues | Round 1 (N=11) | Round 2 (N=10) | |
1. | Too much focus on costs | 2.7 | 2.0 | |
2. | Feasible requirements | 6.5 | 5.0 | |
3. | The vendors don’t get to show their qualities | 6.5 | 5.1 | |
4. | Change of work processes and benefits realization | 6.6 | 5.5 | |
5. | Cooperation between different stakeholders | 7.3 | 6.0 | |
6. | Partnership and innovation are hindered | 9.0 | 7.8 | |
6. | Complex regulations | 8.5 | 7.8 | |
8. | Procurement competence | 8.8 | 8.6 | |
9. | Weighing / prioritizing the assessment criteria | 9.1 | 8.7 | |
10. | Tendering obligations may conflict with long-term planning | 9.6 | 10.2 | |
11. | Lack of coordination and standardization | 10.7 | 11.0 | |
12. | Clear requirements specification | 10.8 | 11.1 | |
13. | Complete requirements | 11.4 | 11.9 | |
14. | Frame agreements | 12.3 | 13.3 | |
15. | Municipal cooperation is challenging | 11.8 | 14.0 | |
16. | Finding and using suitable assessment criteria | 14.7 | 15.0 | |
16. | Integration, compatibility | 13.4 | 15.0 | |
18. | Monopoly-resembling vendor conditions | 14.9 | 15.5 | |
19. | Vendors tend to oversell | 15.5 | 16.9 | |
Kendall’s W | 0.354 | 0.563 | ||
The Kendall’s tau values (Table 2) shows the similarity between the three panels. The correlation was statistically significant between procurement officers and HRISOs, with a value of 0.450. However, as the value is below 0.5 there is not a high level of agreement and it made sense to have separate panels. The correlation was even smaller between the internal stakeholders (procurers and HRISOs) and the vendor representatives. I will explore the differences between the panels further in our discussion section.
Finally, Kendall’s W values in Tables 3, 4, and 5 indicate the level of consensus between different members of each panel after both rounds of ranking. The consensus increased in all three groups from the first to the second round. Yet another round might have led to a loss of respondents without the consensus increasing all that much.
Discussion on findings is done in light of previous research and whether prior findings (Table 1) are confirmed. More importantly, various new challenges and negative impacts are identified that have not been highlighted before.
Several of the main findings relate to stakeholder critical issues. The different stakeholders had differing views on the procurement challenges, the Kendall’s tau values showed clear differences between the three panels. This difference between the stakeholders may in itself be a challenge. If I had included internal users as yet another stakeholder group in my panels, I might have found further differences.
It is also found that vendors ranked the critical issue of cooperation between stakeholders among the top five challenges. This confirms previous findings on stakeholder critical issues being important in IS procurement & implementation.
The panels did not highlight any critical issues related to information asymmetry with consultants, even though I asked for challenges in procurement of information systems and IS services, including consulting. The critical issue “Vendors trying to oversell”, may be related to information asymmetry. This was in the narrowed down list, but it was ranked consistently low. On the other hand, my findings did not suggest that gaps between stakeholder goals and project goals were a challenge.
The data suggested that balancing between different objectives and goal ambiguity is a challenge. The terms were not used in the consolidated list, but vendor respondents point to feasible requirements, i.e., customers are asking for more than they plan to use, as one of their top challenges. According to one vendor, this challenge is due to “Many stakeholders being involved in the early parts of the procurement and implementation process; they all have their wish list, and no one takes charge of prioritizing and shortlisting”. I did not find support for favoring of local vendors and minorities as a challenge. This was surprising, as the vendor representatives in the sample were mainly from the big national vendors, and should be inclined to bring up the critical issue if they felt it caused them to lose contracts. Partnership and innovation was also an critical issue, especially for the HRISOs and the vendors. Transparency for ensuring fair competition between vendors is clearly a public-sector-specific challenge; private firms can be more pragmatic on these critical issues.
The results of findings confirm “Specify requirements before announcing tenders” as an important challenge. Our panelists have used other denominators that are clearly linked. They see developing “Clear requirements” as one of the key challenges. The regulations normally require procurement entities to develop requirement specifications without talking to vendors. A procuring entity may have limited knowledge of what to ask for in a niche area. And they are dealing with experienced vendors who know their software. This challenge of developing “Clear requirements” is rated high both by procurement officers and by HRISOs . Our panelists also brought up the critical issue of “Complete requirements,” which they ranked slightly lower.
The vendor panels had a slightly different view on the challenges, highlighting feasible and realistic requirements from their customers, but were less concerned with getting the specifications completely and clearly. It may not be in all vendors’ interest to have clear and complete requirement specifications, as this may give them less leeway when creating their bid.
A main challenge for the vendors is rather to get an opportunity to show their qualities. The very detailed requirement specifications would limit these possibilities. Vendors viewed “Focusing on initial procurement costs instead of life cycle costs” as the top challenge, hence the previous finding (Tysseland, 2008) is supported. In the brainstorming some of the panelists explained this challenge and the consequences of not solving it.
According to one, the inherent processes in the systems are not evaluated as part of the selection. Only costs for investment, user support and maintenance.
There are some interesting new findings in the lists of top challenges (tables 3-5). Experts across all three panels rated the critical issue of facilitating change in work processes and benefits realization as the most important procurement-related challenge (1, 7, and 4). This finding supports the benefits realization literature (e.g., (Ward & Daniel, 2006), which highlights the importance of planning from early on for benefits from IS investments. IT managers actually ranked change management of work processes and benefits realization as the top challenge. This may be somewhat surprising, as the change of work processes starts after a contract is signed and the responsibilities of the IT personnel are finished. The critical issue has not been identified in previous literature. The results indicate a need for further research and for education on benefits realization practices in connection to IS procurement in the organisation.
The critical issue of technological integration and compatibility of purchased systems was ranked third both by the IT procurement managers and the HRISOs. This is a technical challenge, relating to questions like interoperability. Lack of integration results in silted systems. Interoperability has been high on the agenda in the organisation, and it is believed to be the most critical issue facing businesses that need to access information from multiple information systems (Park & Ram, 2004). Multinational organisations tend to have a large amount of information systems covering the needs of very diverse sectors.
Lack of coordination and standardization of the procurement and implementation process was ranked as the 4th most important challenge by both procurement personnel and HRISOs. In order to understand this critical issue we have to take into account the sample in these two panels, which were largely made up of employees in municipalities.
Finding and using good assessment criteria and weighing/prioritizing the assessment criteria were also high on the agenda of the internal stakeholders. This may be related to the need to stick to the requirement specifications due to the formal tendering process, and to the possibility of vendor complaints. In addition, rules and regulations were seen as hindering longer-term vendor-customer partnerships, both by HRISOs and vendors. Longer-term cooperation could give some benefits such as less scope for opportunism from the vendors (Parker & Hartley, 2003), and trust relationships and coordinated strategies between buyers and suppliers (Parker & Hartley, 1997).
Category / critical incidents | Challenge / negative Impacts on stakeholders | Explanation | ||||
1. Requirement specification | ||||||
Quality | 1.1 | Clear | Difficult to define clear and objective requirements. | |||
| 1.2 | Complete | Incomplete req. specifications | |||
| 1.3 | Feasible | Customers ask for more than they plan to apply | |||
Content | 1.4 | User support as part of the requirement specification | Get optimal user support from the vendor | |||
| 1.5 | Operations as part of the requirement specification | ||||
| 1.6 | Requirement for specific technologies | Require for instance ASP or cloud computing | |||
Process for developing the req. specification | 1.7 | Based on process improvements | Make a requirement specification based on, e.g., a process map or use-cases | |||
1.8 |
| Verified requirements specification | ||||
1.9 |
| Balanced/prioritized between different needs | ||||
1.10 |
| Allocation criteria | Difficult to develop criteria for allocating contracts | |||
2. Change management | 2.1 | Change of work processes and benefit realization | Difficult to achieve change of work processes and of the organization and to realize the possible benefits | |||
| 2.2 | Resistance to change | ||||
| 2.3 | User training for new systems and work processes | The need for training is not estimated properly | ||
3. Different stakeholders, cooperation | 3.1 | Involvement of procurement personnel | Procurement of information systems may be done without involving the group with procurement competence | ||
| 3.2 | Gathering of key personnel for the procurement process | Gather personnel with the critical knowledge | ||
| 3.3 | Cooperation between different stakeholders | Different government sectors or business units have to cooperate, without understanding each other´s needs | ||
| 3.4 | Differing viewpoints and interest in assessment criteria (of the vendor) | Need to find common criteria | ||
| 3.5 | Conflict | Conflict between different business units | ||
| 3.6 | Citizen focus | Not enough concern for “customers” | ||
4. Competence | 4.1 | Procurement competence | |||
| 4.2 | Competence in rules and regulations | |||
| 4.3 | Judicial competence | |||
| 4.4 | Financial competence | |||
| 4.5 | Competence in negotiations | |||
| 4.6 | Product competence | |||
| 4.7 | Competence in license critical issues | |||
| 4.8 | Domain competence | |||
| 4.9 | Competence in existing systems and infrastructure | |||
| 4.10 | Competence in installation, testing and supplier responsibilities | |||
| 4.11 | The supplier’s competence | |||
5. Competition | 5.1 | Lack of methods for evaluation | |||
| 5.2 | Find good criteria for evaluation | |||
| 5.3 | Weighing/prioritizing between different assessment criteria | |||
| 5.4 | Comparing systems | |||
| 5.5 | Conditions resembling monopoly situations | Only a few vendors of the requested system type | ||
| 5.6 | The supplier is not given the opportunity to show their qualities | The customer asks so that the vendor does not get the opportunity to show their competitive assets | ||
6. Contract critical issues | 6.1 | Complexity, few complete contracts | Difficult to calculate the cost of all items specified in the contract | ||
| 6.2 | Lack of use of the government’s standard contracts | |||
| 6.3 | The government’s standard contracts | These differ from traditional contract regulations (rules, laws) | ||
| 6.4 | Unclear contract, differing understanding of contracts | Unclear if certain critical issues are included in a contract | ||
| 6.5 | Contracts with duration over several years | Discounts included in longer contracts | ||
| 6.6 | Frame agreements | Frame agreements that ensure flexibility or that have price mechanisms which are beneficial over | ||
Implications for research and practice
The classic challenge of coordinating between various stakeholders in IS procurement and implementation and IS investments emerged as one of the major challenges. Research results support previous calls for more focus on managing these challenges in procurement and implementation practices and processes.
The experts further highlight the importance of clear, complete, and feasible requirements specifications. Increased focus on requirements specifications may be especially important for the organizations. Our data confirms that this dilemma still has to be solved in an organisation. One possible solution could be more use of competitive dialogue, where vendors are invited to participate in a competition and in a dialogue with the procuring entity before the requirements are fully specified.
The challenge of technological integration and system compatibility highlights the importance of involving IT expertise in the procurement and implementation process.
This Delphi study revealed typical challenges for IS procurement and implementation to various stakeholders in an organisation. Three expert panels defined 98 challenges and dilemmas, divided into 13 categories: requirements specification, change management, cooperation among stakeholders, competence, competition, contracting, inter-municipal cooperation, governmental management, procurement process, rules and regulations, technology and infrastructure, vendors, and IT governance. The results provide a rich overview and complement the previous, largely conceptual and case-based literature on public IS procurement challenges.
The study supports previously identified challenges related to stakeholders and to balancing between their objectives related to requirement specifications. All relevant stakeholder groups should be involved in procurement projects. More research is needed on critical issues such as stakeholder management and on balancing different goals without asking for more than is needed. The interplay between procurers and vendors in public procurement has not previously been much researched. This interplay may not function very well in public sector due to recurring competitions and complex regulations.
One especially important critical issue is the conflicting interest of IT procurers and vendors. Procurement and Implementation personnel strive for complete and clear requirements specifications, at the same time vendors seem to prefer less detailed specifications this would give them more room for showing qualities that are not mentioned in the request.
In addition, the study revealed challenges that have not been discussed previously in connection to IS procurement and implementation, such as aligning benefits realization to procurement. The study further supports previous findings on plain focus on costs. If procurement managers and HRISOs want to achieve benefits from investments in new systems, they need to balance the focus on cost with the need for quality, and they need to give room for vendors to show their qualities.
The challenge of complex and constraining regulations was also prevalent. This may make the process more complex and costly than needed, and may also hinder SMEs from participating. Lack of coordination and standardization was also revealed. Public procurement of Information Systems is a complex task, and many years can go by between subsequent projects in one professional domain, before new systems are bought, hence help should be needed. The problem could be overcome by copying successful procurement processes from other government entities or collaborating municipalities. However there may be risks with in doing this.
Further work will also focus on creation of cause-effect relationships between the most commonly observed critical issues through qualitative analyses of the brainstormed data and through additional fieldwork. Another natural avenue for further work resides in cross-country studies, which might reveal more information about generalizability of these results to other countries with equally strict procurement regulations.
References
Dawson, G. S., Watson, R. T., & Boudreau, M.-C. (2011). Information Asymmetry in IS Consulting: Towards a Theory of Relationship Constraints. Journal of Management Information Systems, 27(3), 143-178.
Flak, L. S., Nordheim, S., & Munkvold, B. E. (2008). Analyzing stakeholder diversity in G2G efforts: Combining descriptive stakeholder theory and dialectic process theory. e-Service Journal, 6(2), 3-23.
Flak, L. S., & Rose, J. (2005). Stakeholder governance: adapting stakeholder theory to the e-government field. Communications of the Association for Information Systems, 16, 642-664.
Gupta, U. G., & Clarke, R. E. (1996). Theory and applications of the Delphi technique: a bibliography (1975-1994). Technological forecasting and social change, 53, 185-211.
Johnson, C. W. (2011). Identifying common problems in the acquisition and deployment of large-scale, safety-critical, software projects in the US and UK healthcare systems. Safety Science, 49(5), 735-745.
Murry, J. W., & Hammons, J. O. (1995). Delphi: a versatile methodology for conducting qualitative research. Review of higher education, 18, 423-436.
Okoli, C., & Pawlowski, S. D. (2004). The Delphi method as a research tool: an example, design considerations and applications. Information & Management, 42(1), 15-29.
Pan, G. S. C. (2005). Information systems project abandonment: a stakeholder analysis. International Journal of Information Management, 25(2), 173-184.
Park, J., & Ram, S. (2004). Information systems interoperability: What lies beneath? ACM Transactions on Information Systems (TOIS), 22(4), 595-632.
Parker, D., & Hartley, K. (2003). Transaction costs, relational contracting and public private partnerships: a case study of UK defence. Journal of Purchasing & Supply Management, 9(3), 97-108.
Poon, P.-L., & Yu, Y. T. (2010). Investigating ERP systems procurement practice: Hong Kong and Australian experiences. Information and Software Technology, 52(10), 1011-1022.
Rosemann, M. (2003). Procurement. In H. Bidgoli (Ed.), Encyclopedia of Information Systems (pp. 505-516): Elsevier Academic Press.
Rowe, G., & Wright, G. (1999). The Delphi Technique as a forecasting tool: critical issues and analysis. International Journal of Forecasting, 15, 353-375.
Rowley, J. (2011). e-Government stakeholders—Who are they and what do they want? International journal of Information management, 31(1), 53-62.
Ryan, C., & Walsh, P. (2004). Collaboration of Public Sector Agencies:Reporting and Accountability Challenges. International Journal of Public Sector Management, 17(7), 621-631.
Saarinen, T., & Vepsäläinen, A. P. J. (1994). Procurement Strategies for Information Systems. Journal of Management Information Systems, 11(2), 187-208.
Schmidt, R. C. (1997). Managing Delphi Surveys Using Nonparametric Statistical Techniques. Decision Sciences, 28(3), 763-774.
Ward, J., & Daniel, E. (2006). Benefits Management: Delivering Value from IS & IT Investments. Chichester: Wiley.
