"To make systems safe, we need to look for ways to constrain their freedoms; bring them back to the level of simpler purpose-built devices." #readingToday
To make information-responsive systems safe, we need to look for ways to constrain their freedoms and to bring them back to the level of simpler purpose-built devices, where intrinsic outcomes can be assured. Some software systems are being hardcoded into specialized chips, which suffer from the same problems and are even harder to correct. We'll need design engineering to either filter out certain messages before they can be heard from the inputs of software, (as one filters out noise in audio), or to respond to every possible message that arrives with a suitably stable response – and on a timescale commensurate with their appearance.
In Search of Certainty, Mark Burgess, 2013