New Zealand Blood Service Team Describes the Importance of Identifying Business Need and Involving Business Owners When Implementing Application Changes

The healthcare world is changing rapidly, and change includes ever more pressure to accurately and cost-effectively validate and test software used in laboratory settings, patient management systems and every other aspect of healthcare IT.

We're constantly striving to better understand the stress points and gaps that block the path to success for healthcare organizations.

Art Trevethan, SmarteSoft Director of Strategic Development, recently had a discussion with John Cox, Project Manager, and Be Couper, Test Manager, with New Zealand Blood Services (NZBS), about IT challenges faced by blood banks - and more generally - healthcare organizations. The group discusses the importance of thorough planning and excellent organizational communication methods and the importance of identifying business need for IT change. Involving business owners in the change process is a critical factor in delivering a quality product on time and on budget.

Art: First things first, just to set the stage could describe your areas of responsibilities?

Be: I'm the Test Manager for the eProgesa upgrade project at New Zealand Blood Service.

John: I'm responsible for the entire breadth of the project which encompasses a number of different work streams. My responsibility is making sure at the end of the day that we deliver on what we promised in the business case.

Art: As you are performing your responsibilities, what are some of the major challenges that you face day-to-day or as part of the project as a whole - what are the major challenges that you face in your role.

Be: For me it's been geographic disbursement of the project team, in particular the testing. The testers are from the business and weren't actual testers. They had to be trained to be testers. And then they are not in Auckland with me - three are in Wellington and another three are in Christchurch. So it took an effort to get the cohesiveness of the team all going in the same direction because of us not being together. That's been one of the biggest one's for me.

John: I think integrating the whole validation regime into the other project processes - be it redesign of operational processes, change control, the raising and management of issues - has actually been quit a big challenge as well. Because we have different people responsible for each of those different areas making sure that we've got the appropriate communication channels and feedback loops in place have been a very complicated thing to get up and running. And then making sure that through all of that we are able to involve our business owners and business representatives in reviewing various bits and pieces of the process as appropriate has been a tricky thing to integrate as well.

Art: So it sounds as if I'm hearing a very strong message that communication is a very important part of what you're trying to do?

John: We started out with a good six months worth of planning and I think during that time we were quite clear about what our works streams were going to be and what each person was going to be responsible for. And I think as the project team has increased in size - every time you add a person on you add a factorial number of communications and it does become more complicated because we have to spend more time making sure that people understand who they are responsible to communicate with. It's really been a bit of an evolutionary process rather than just saying here's the process, do it. It's been a case of here's a bit of the process, let's see how that goes, now let's build the next bit and then let's build the next bit so now we're to a point where we have six or eight people who are effectively singing from the same hymn sheet and are quite clear about what we need to communicate. We've got two or three main databases which are capturing various aspects of the data. And we're managing those information flows now reasonably well.

Art: Given some of your previous experience, perhaps outside of New Zealand Blood Service, is this a common challenge that you see in other organizations that you've been involved with or is this something that you feel might be specific to this organization or project?

John: I think it is a common theme. One of the complexities we have within New Zealand Blood Service is that we are a regulated government agency so we are under a good deal of scrutiny from the government and the licensing board so there is a very high expectation around the quality of the product that we deliver. There is a high expectation around the documentation that supports that delivery. And I think in many projects you can slip a little bit and you can save yourself a bit - it doesn't matter, it will come out in the wash eventually. But in this project for those regulatory reasons we can't really do that. We have to get it right first time.

Art: So because of the regulatory environment that you're in you've got additional constraints that perhaps make some of these things even more critical to iron out.

John: That's correct. The regulatory environment adds to our overall desire to deliver a quality product and makes us very focused on making sure that when eProgesa goes live it goes right the first time.

Art: So as you are going down this path you obviously have a set of goals that are set. Are you seeing impact factors from the rate of change in the technology available?

John: That's an interesting question. I don't think so too much. And that's not to say the technology hasn't evolved in the time we've been working on the project but we've been very careful to make sure that we chose the latest technology at each point and that effectively we've stuck with that technology - we haven't been seduced into upgrading or updating it anymore than is absolutely necessary to keep all versions of the software current, for example.

Be: And we've been using the latest technology to communicate with each other and to support the project processes.

John: Yes.

Be: Using SharePoint and Skype and GoToMeeting as well; so you know from that point the project's been quite different in managing the communication and how we do communications through our web.

Art: What are the bodies that have an effect on what you're doing there in New Zealand?

John: The primary one is a government organization that is responsible for giving New Zealand Blood Service its manufacturing license. We have an accountancy firm monitoring project quality assurance. We also have the Office of the Auditor General doing an audit of the organization as well, specifically of the project. And we have quite a few different organisations that have a degree of interest in what we're doing with the project.

Art: So how does that affect your risk management or risk tolerance within the organization, as you're adopting a newer technology, for instance - the eProgesa system, and some of the compliance regulations that you have and budgetary concerns? How do you deal with the risk management and tolerance level?

John: I think that right from the outset there has been recognition that this project is a very high risk project even though the scope is largely like-to-like replacement in terms of functionality. We've still effectively putting in a brand new system and everything that that entails. So we have been asked to manage the project in a very careful way to make sure that we have clear processes, good documentation, and a high level of business involvement in the project. We have talked about the testers that come from the business. We have two subject matter experts who come from the business. We have a half a dozen of what we call business owners to represent the business at a management level and we get our governance through a steering committee, which are also largely members of the executive. So I think there is a recognition of our project as very high risk and our processes are approached it that light - the amount of quality that we have applied, to the involvement of staff, all contributing to the mitigation of those risks.

Be: One of the things we do as well is we comply or adhere to GMP 5 (Good Manufacturing Process) which is a manufacturing process. And their recommendation is risk based validation. So we look at our project from that perspective.

Art: Would you say that from the corporate culture, the top level managers, have they been heavily involved with a sign-off budgetary process and provided the kind of resources and support needed?

John: Yes, absolutely. Obviously the project started with a business case which they were instrumental in signing-off. And that business case was based upon a number of different strategy documents and various different work streams. Each aspect of the project has its strategy document and those were prepared in consultation with the business owners and have been signed-off by the steering committee and I guess indirectly by the CEO and board as well.

Art: And is the steering committee made up of executives or industry specialists?

John: It's made up of NZBS executives and also we have somebody from our IS outsourcing partner but largely its internal.

Art: My question was directed at: do the government bodies have a voice on that steering committee?

John: No they don't. They largely maintain their independence in that respect.

Art: Let's take a moment to talk about some of the day-to-day challenges that you face. How large are you teams?

John: We have six people on our project management team and then Be has a further six people who make up our validation team. We will shortly be recruiting four people who will do our documentation and training. And we have, probably, about half a dozen people on the IT outsourcing side. And in addition to that there would be another six or eight people who would be involved in various other technical work streams. From now until early next year there will be 30 or 40 people working on the project.

Art: You both mentioned earlier that the geographical displacement of the team led to some communication challenges. How many different locations are there for the people you described?

Be: For the testers and the subject matter experts, they aren't in Auckland, they're in two different locations as well - so at the moment there are five.

Art: And I'm going to ask each of you this separately because I think that you've got some different teams. Be, what kind of skill sets are you looking for in your perfect team member?

Be: To have business knowledge; that's key. To know the testing lifecycle. To be able to do documentation. And verbal and written skills need to be excellent. Team effort and participation is important and to be able to work autonomously because I'm not right with them all the time. To be passionate about working at NZBS.

John: Detail, process focused.

Be: Yes.

Art: And John, I'd ask you the same question because perhaps you have different people for your team. What skills sets and qualities would they possess?

John: I think primarily I want them to be accountable in the areas they are working in and to champion those areas. It's a case of each person having work streams defined to them and then making sure those work streams get delivered on time and to the quality that we expect. And to make sure that they are clear about what delivery actually means in terms of documentation or processes or training or what have you. And in addition to that I need them to be able to think and work aside the square a little bit as well because no one is working alone. So they need to be aware of what others are working on as well and how they interact with those people. The ability to work as part of a team and to understand their role within the team is important. And then beyond that it really is just individual capabilities and abilities to be able to do the jobs - things like ability to plan, ability to communicate verbally and through writing.

Art: As you're doing this, how do you initiate an application change?

John: An upgrade?

Art: Yes, in this case it's an upgrade.

John: An IT project should be initiated by the business. They should have identified a business need to upgrade and if the product is meeting their requirements in every way, shape and form then I don't believe an upgrade should occur unless you've got some clear drivers behind it. In this instance there were a number of clear drivers for doing the upgrade. Then really it is a case of if we are going to do it, what is the scope of the work, making quite sure we are clear about what we want to achieve, what the outcomes are and then initiating the project from there.

Art: Along with that, the initiation of a process, what would you say is your timeline, beginning to end, for this type of a project in your environment?

John: It's about 2½ years but we spent a good deal of time planning this. The environment here in New Zealand is that we get to put forward a business case which will ask for a specific amount of money and it's very unlikely we'll get any further funding beyond that - so that business case has to be right the first time.

And so, in the first instance, we have to do a fair bit of work to define the scope and secondly, what are the costs to deliver on that scope. Then the case goes for approval.

Art: So as you are going through this process I'm sure there are numerous methods that you've used, how do you manage these processes?

John: Through a lot of discussion and debate, I think. We had an internal conference at the weekend and one of the presenters was talking about what he was calling "hot" teams and the characteristics of the so-called hot teams. One of those characteristics was the ability to talk about the hard things and I think that is one of the things we've done reasonably well because we have a lot of really robust discussion about the processes and make sure we get the appropriate people involved and then once we've made a decision we can document that. So we have a consistent understanding of what those processes are and who is involved and in what way. And I think it's probably true that the regular dialogs that we have help us come to a common understanding as a team about those processes. It's quite an organic thing rather than one person just saying here it is, follow the flow chart.

Art: That's fair. And would you say, Be from your view as far as the processes go and managing that large project - do you follow the same path?

Be: Yes, there are some processes that we are following to the T from a test lifecycle but others have to be flexible and if it doesn't work, then look at it and readjust.

Art: Of course, within this process there is an end goal, but along the way I am certain there might be what you consider "wins." What constitutes a win for your team?

John: Well let's go back to that conference that I just referred to. It's interesting that one of the other things that this chap said about hot teams was that they celebrate their wins regularly and frequently. We've just had a very successful process audit which is encouraging and so we celebrated. We've been out for meals on an ad hoc basis.

But I think from my perspective, one of the challenging things is actually to identify a win when it occurs because it's so often couched in so many activities which are going on. So we do get to certain points in the project were we get something signed off or we have a successful steering committee meeting and things like that which constitute wins.

Be: Well, I ought to look at it from the perspective of the project when we finished a phase like we did - the recent release and acceptance - and that went really well, the first time the team had done a really big piece of work with eProgesa being installed. That was quite a big win for us when we finished that phase with eProgesa - not necessarily how many defects there are because that is out of our control but how we managed that phase.

Art: If you are working on a project that extends over a 2½ year period there has to be subsets of unit progress. How do you monitor your progress against a 2½ year goal? How do you know if you're on target, if you're 10% complete?

John: The short answer to that is good planning. We have a project schedule that we are tracking our progress against which effectively works backwards from our goal update in August next year back to today. And as we mentioned before we have a number of significant milestones over that period of time and collectively we work toward those milestones.

Art: As you're going through this project you've had a number of challenges to overcome. What have been your biggest wins so far?

John: I think, obviously, getting the business case approved considering that had to go all the way to the ministry level was a big win for us. Getting such a good project team together and effectively choosing half a dozen individuals and our ability to work together has been a big win. I think connecting some good software tools has been a big win for us because they have simplified and to an extent complicated what we are doing. And I think probably just having a common understanding about producing quality output and delivering on time - and continuing to do that - is a big win for us as well.

Art: Be, I'd look for your perspective on that as well, on what have been your big wins along the way.

Be: When I first started on the project I worked across a couple of areas and that was testing and configuration management and John has already mentioned getting the software tool. It made a huge difference for us having really good software tools that can facilitate our processes and make them easy and robust. Getting the team together, as John said, is important as well. You don't know what you're going to get but the team works really well together and enjoy what they are doing and are passionate about getting it right and doing a good job. So that's a really big win for me - that they're all so positive about it. And likewise, integrating my process with the other project team members and getting those ironed out and, for me, to get the loop back into validation that I need has been great.

Art: How would you compare this to other projects you've been involved with in the past? That's a big, wide question so you've got some leeway here.

John: I think also one of the big differences is that in proportion to the size of the organization the project that we are working on is huge. If we miss milestones or under-deliver product than the whole organization feels the impact whereas historically I think many of the projects we have worked on have affected a relatively small part of the organization - a single department or a single division. Therefore if the project fell short of expectations it would affect just that part of the organization - a relatively small part of the organization. So the profile of this is of national importance and consequently that puts a lot of pressure on you to deliver the right thing the first time. To me, that's probably the biggest difference. It's a high profile, high risk project.

Art: Let's talk a little bit about some of the tools. How would you tie in the importance to the overall scope and scale of the project - what is the importance of regression testing efforts?

Be: It's vital. It's to make sure that nothing is broken when new patches or fixes go in. Our testing is program based and we will know which program has been effected and we will regression test all of our test cases that are related to that program.

John: I think in this project the regression testing is as important as is the validation in the first instance. If you had a vendor who was delivering high-quality software that is largely bug-free - not that such a vendor necessarily exists, I have to say - you might worry less about regression testing. You could say that if it's right after a first round of testing than we won't worry so much about regression testing. But for a lot of the quality discussions we've had, the regression testing is as important as the initial testing.

Art: For your documentation, what are you required to provide to the various regulatory bodies?

John: The areas where they are most concerned are obviously around validation. And they will certainly want a good deal of detail around what was tested, how it was tested, what the outcomes were, how the defects were managed and resolved. And the other area where they tend to focus a lot is around training - training documentation and evidence that training has been completed. And probably a third area would be the impact that the upgrade would have on business operations - making sure that there aren't any additional risks or compromises in terms of the quality of the products that we produce.

Art: Anything that you can add, Be?

Be: The test plan and test executables - what are we testing, how we are testing, what are the variances, what are the exceptions, defects that are outstanding and how are we going to manage those. The big thing they want to see is the traceability matrix. Making sure all our requirements have been covered and have been tested.

Art: How do you approach that documentation that is required? How do you create it? What methods do you use for that?

Be: We've got templates that we've designed. And it defines our approach and how we are going to achieve it.

Art: The design of those templates, was that done early on in the process in relation to the requirements that the regulatory bodies had or is this something that has evolved through the process?

Be: We started early on, not only based on what the regulators required but based on past experience I've had. And they were signed off by the business owners so they approved what was in them.

Art: Is there anything that you'd like to make sure that the readers of this article should know? What would you like them to walk away with?

John: I think not being afraid to embrace new technology as a part of a project like this. And by that I mean the benefit that we've got, not only from SmarteQM and other tools, but we've also introduced a configuration management database. We're looking at a tool to author our documentation and also a tool to help us with eLearning delivery. Those things I think will really increase our productivity. There is a steep learning curve associated with any of them but the payback is actually huge not only for the project but also for the organization because many of these tools we'll be able to leave with the organization to use in the future.

That's one of the wins from the project and I think it really has helped us to maintain the quality of the project. But quite frankly, without a testing tool I don't know how we would have ever managed the quality and the extent of the testing we need to complete this process. And it's the same for the other tools as well. In hindsight we wonder how on earth we would have managed with a spreadsheet or a table in a Word document.

And also the value of planning - we spent a lot of time planning the project and making sure we were quite clear on what our work streams were, what our timeframes were, what our costs were, and what our deliverables were. That was done over a year ago and we're still working to those timeframes and deliverables and that gives us good credibility not only with our regulators but with the executive as well.

Probably a third area would be around involving the business. We hear so many lessons learned from other blood centers and other organizations - wishing in hindsight that they had involved the business more. That was certainly something that Be and I started out this project making really clear to the steering committee, that we needed a high level of involvement from the business. And we've achieved that through our accreditation team, through our testers, through our business owners and so on. We have effectively only two people from outside the business working on this project - that's myself and

Art: And Be is there anything that you would like for the audience to take away from this?

Be: I think John has said it all, really. I'm just so pleased that we got the tools. Honestly it just makes a huge, huge difference in our working day.

Art: Thanks for sharing this information. We appreciate your time and expertise.

Click here to download a pdf of this interview.

We're Interested in Your Perspective on the Healthcare IT Arena

We want to hear about your experience. What IT issues cause recurring pain? If you've solved an IT challenge, you can share it with the healthcare community here. Contact us. We'll help you spread the word.

We're Proud of These Success Stories

SmarteSoft has helped hospitals, blood banks, medical device manufacturers and other healthcare product and services providers achieve software success. Satisfied clients include Hill-Rom, Ohio Health, Scottish National Blood Transfusion Services, Clear Trial, American Red Cross, the US Department of Health and Human Services and New Zealand Blood Services.

At SmarteSoft, we're committed to understanding the challenges healthcare professionals and healthcare software vendors face. Data validation and application testing of the software you use is as important as improved speed, accuracy and cost of test.

For a free consultation call us at (512) 782-9409 or drop us a line at healthcareinfo@smartesoft.com. We'll review your challenges and help formulate an effective test strategy.