I recently posted a link to a news article with the title “Spreadsheet error led to Edinburgh hospital opening delay”, which you can find here. There was a great response to this post so thanks to everyone who interacted with it.
Flippantly I suggested the experience of this project was good evidence for adopting a Model-Based Systems Engineering (MBSE) approach. I say flippantly, not because I don’t believe it to be correct, but rather, as several people pointed out, it’s never as straight forward and as MBSE = good, Documents (in this case Spreadsheets) = bad. Some of the themes that arose during these interactions require a more in-depth discussion, and so I will try and address some of them below:
If everything is a model, then nothing is a model
One of the ways MBSE is sometimes misunderstood is that people (quite rightly) say “The information in the spreadsheet is a model, so this is MBSE”. The first part of this is correct. If you use the term ‘model’ in a general sense, then the spreadsheet is a ‘model’. MBSE, however, doesn’t use it in this way, in MBSE the ‘model’ has an important property, it is constructed using a defined modelling language which has well-defined semantics and syntax. Secondly, the modelling tool is aware of the syntax of the modelling language. In MBSE there is a whole class of syntactical errors that the modelling tool simply won’t let you make.
As a trivial example of how syntax enforcement can help, languages like SysML don’t allow two elements, of the same type, and with the same name, to exist within the same namespace. In spreadsheet terms, this is similar to saying, on any given tab, if you specify something on one line, i.e. “The max value of x = 4352 units” it won’t let you re-specify it again on another line. Why is this important? Well for small to medium sheets, probably not, but consider large sheets. How do you know if the information on line 22 is duplicated by the information on line 2563? Clearly, this problem increases with scale. Providing these two values are the same then you don’t need to worry, but what if one value changes? How do you even know there are two entries, let alone whether they are still consistent?
This example is only one small difference, and nobody is claiming just removing this potential problem leads to success. Once you draw it to peoples attention, they quite rightly point out that there are things you can do within the spreadsheet to mitigate it.
The Problem With Spreadsheet-Based Solutions for Spreadsheet-Based Problems
It has been pointed out that you can avoid some of these problems with spreadsheets by constructing better spreadsheets. This claim may be true; I’m sceptical though because every time you build something into your spreadsheet which makes it more like an MBSE model, you’re making it less like a spreadsheet, and I think you’re effectively adding technical debt. Peter Senge’s first law of The Fifth Discipline is “today’s problems come from yesterday’s solutions.” But in any case, Document-Based approaches aren’t about a single spreadsheet; they are about multiple documents. So how do you also address the equivalent issues within Word, Powerpoint, Visio or whatever document-based tools you use? Also, it’s not just a problem within a document it’s a problem between documents – let’s say you eliminate duplication in a spreadsheet, how do you prevent the risks of inconsistnecy between a spreadsheet and a Visio document? They are, by design, intended to be unique data sources.
Anything You Can Do I Can Do Worse
Another example of the differences is the MBSE concepts of Definition and Usage. These concepts mean that you can define something once (a definition) and use it in multiple places (usages). Because it’s only specified once there’s only one place to change it, and doing so changes it everywhere it is used. You may suggest this is just the equivalent of putting something in a spreadsheet cell, and referencing it in lots of places, however, it’s more sophisticated than that. Firstly the ‘thing’ can be more than a single cell; it can be a whole structure. Secondly, the tool knows everywhere it is being used – so if I want to change it, I can first check my changes won’t break any of the current usages. Thirdly, it does something called type checking. Spreadsheets do a degree of type checking as you will know if you’ve ever used some text as the input to a function expecting a number! SysML (and supporting tools) go one step further. Useful SysML tools will tell you if you are trying to use a value of pressure in Pascals when the formula expects one in Atmposhperes. There’s a great example of this benefit, and the problem it would have avoided with the “Mars Climate Orbiter Units Mismatch” by Michael Vinarcik.
And Now For My Next Trick
As well as removing the potential for some inconsistencies, type-mismatch and syntax errors, MBSE also facilitates some features not easily achievable with documents:
You can query the entirety of the model. Contrast this with a set of 100 files where if you wanted to find something you would have to search each one individually – assuming you knew about and could see all 100. What about the ones on Steve’s C: drive that he hasn’t yet committed to the document management system? (Not wanting to single out any Steves – it’s just the nature of working with documents)
You can perform transformations of the model which means that you can generate other models and documents from it without the introduction of errors. One misconception about MBSE is that all documents disappear, they don’t, they are just not the primary source of information.
I’ve only scraped the surface here, but I’ll summarise it with the following sketch which says that while Document-Based approaches and MBSE both have more useful and less useful features (in the pursuit of quality), there are more favourable features and less troublesome features with MBSE
The Bigger Picture
So far, we have only considered the rather technical details of tools and languages. As several people pointed out, the root cause in the original article is probably one of process. You may even argue that a problem rooted in the process is a problem rooted in the culture. I might agree. There are two things to say here:
- MBSE also needs a robust and capable process
- The process is never enough
Quality, very much like Safety, requires us to not only have a good process, but also an appropriate culture. MBSE is no substitute for these, but if we are working at height (i.e. there is a significant risk), MBSE can act as a guard rail. It won’t stop you throwing yourself off, and to be effective you have to work safely and intelligently, but it can help with unforeseen slips.
For those that are unaware, the title comes from this.
