What is a distributed autonomous system

Find increasing autonomous systems also used in medicine. These autonomous systems include individual medical devices such as surgical robots. But combinations of individual (medical) products also form autonomous systems.

It is not clear to many medical device manufacturers and hospitals

  • Which opportunities offer them these autonomous systems,
  • Which regulatory requirements you have to pay attention to
  • Which Risks have to master them, which are not relevant for many other products.

1. What are autonomous systems?

a) Examples

The most frequently cited examples of autonomous systems are self-driving cars. They are able to safely steer the vehicle to a destination regardless of human intervention.

Household robots and many industrial robots also belong to the class of autonomous systems.

Autonomous systems also exist in medicine, for example

If you want to find out more about these systems, the publications are for you Autonomous Systems in Anesthesia: Where Do We Stand in 2020? A Narrative Review and Robotic anesthesia: not the realm of science fiction any more recommended.

b) definition

A system is referred to as autonomous if it can achieve a given goal independently and adapted to the situation without human control or detailed programming.

Source: Autonomous Systems Expert Forum in the High-Tech Forum: Autonomous Systems - Opportunities and Risks for Business, Science and Society. Long version, final report, Berlin, April 2017

There are also other definitions, because “autonomous” only means first of all independent of or independent. The respective definitions always reflect what is currently important in a particular system, for example

  • independent reloading with mobile robots,
  • Flying without detectable remote control signals with military drones.

An essential feature of autonomous systems is theirs Ability to adapt. This means that they can adapt to different situations independently.

c) Differentiation from automation

In the case of automation, one looks at the substitute, in the case of autonomy rather the independence. The term “automated driving” indicates that driving has been automated. The term “autonomous vehicle” indicates that the vehicle is independent of the driver.

In general, however, autonomous systems are able to work independently even in complex and changing environments. This is with classic Full automation not the case:

  • An industrial robot that has been programmed to place a component on a conveyor belt acts fully automatically. It is not an autonomous system as defined above.
  • An industrial robot that searches for components independently and mounts them on cars, even if there are different cars or the cars are positioned differently and people are in the robot's work area, acts autonomously. It is an autonomous system.

d) Delimitation of closed-loop systems

Many autonomous systems contain closed control loops. The popular terms "Sense Understand Decide Act" (SUDA), "Monitor Analyze Plan Act - Knowledge (MAPE-K) or" Cognitive Loop "are all aimed at the following steps in the control loop:

  • Perceive the situation
  • Make a prediction
  • make decision
  • Implement decision

In contrast to the "classic" closed-loop system, the rules for the prediction and decision in the autonomous system are often not so rigid or not so explicitly formulated, i.e. programmed. Machine learning methods are used regularly.

In addition, autonomous systems are usually also networked systems, because distributed sensors are often advantageous in order to gain a sufficient understanding of the situation. This is not necessarily the case with closed-loop systems.

2. Specific risks of autonomous systems

a) Risks from the autonomy of the systems

Autonomous systems work independently of, for example, human intervention. This often means that the opportunity to intervene at all is lost. This increases the likelihood that a fault in the system will go undetected and lead to damage.

b) Risks from different technical contexts

Autonomous systems are adaptive. So you should be able to adapt to different contexts. These contexts differ in technical aspects, among other things:

  • Quality (bandwidth, latency, availability) and type (different protocols, wired versus wireless Network connection)
  • Type of productsthat are part of the system (e.g. different models from different manufacturers, differently configured products from one manufacturer)
  • Number of Productsthat are part of the autonomous system. This number can change as the operator adds or removes products.
  • Products can also be because of technical problems of the network or the product itself fail, become unavailable, or no longer perform as promised.
  • Physical environment (Brightness, humidity, noise, dirt, etc.)

c) Risks from different clinical contexts

Autonomous systems should also behave safely in different clinical contexts. Attributes of these clinical contexts are:

  • Number, availability and training of staff
  • Mental workload, stress (e.g. due to alarms, shift work, shift changes, emergencies)
  • Number and state of health of patients
  • Availability of aids, medical devices and other systems

d) Risks from adaptive algorithms

Many autonomous systems acquire the ability to adapt to different contexts by using adaptive artificial intelligence algorithms.

Additional information

Read more about the challenges and regulatory requirements for the use of artificial intelligence, especially machine learning, in medicine.

e) Risks from a lack of interoperability

When (medical) products are connected to form an (autonomous) system, the number of interfaces increases disproportionately. There are regular problems, especially at the level of semantic interoperability. This is why the FDA is pushing for international standards to be used.

Additional information

Read more about interoperability here and find out which protocols, formats, taxonomies and nomenclatures are used at the various interoperability levels.

f) Conclusion

The strength of the autonomous systems, namely being able to deal with variable contexts, is also their weakness:

The combinatorial explosion of situations is difficult to keep track of for both manufacturers and operators of autonomous systems. At the time of development, manufacturers cannot yet know all future contexts. The operators, in turn, do not have all the information they need to combine products into systems.

3. Approaches to managing the risks of autonomous systems

a) Naive approach: worst case

The most obvious approach for assessing the risks and later controlling them is to assume worst-case scenarios for the various contexts. Examples are:

  • The patients are in critical condition.
  • The staff is unable to intervene.
  • One or more products fail in the system.
  • One or more products provide data at the edge of the specification range.
  • The network no longer has the necessary bandwidth.
  • The users configure the systems incorrectly.

Such a worst-case assumption regularly leads to unacceptable consequences:

  • The autonomous system may not be placed on the market or put into operation.
  • The performance requirements for the products must be so high that the costs the product uncompetitive do.
  • The scope of application must be like that limited that the benefits of the autonomous system - namely its ability to adapt - hardly come into play.

b) Model-based dynamic risk management

Context-dependent decisions

Model-based approaches (especially model-based dynamic risk management) try to avoid these consequences. This is done by evaluating the decision on the acceptance of errors and misconduct in a context-dependent manner. The following examples should illustrate this:

  • Blood pressure that is 5 mm Hg inaccurate in a stable patient may be acceptable. In the case of an intensive care patient whose medication is controlled depending on the blood pressure, however, not.
  • An oxygen sensor accidentally slipped off your finger may not be a problem if the breathing air is monitored by another sensor at the same time. However, if this second sensor is missing, an alarm is essential.

In order to make such context-dependent decisions, all necessary information about the individual products as well as the context must be available.

Interfaces, models and domain specific languages ​​(DSL)

This in turn requires a standardized, product or manufacturer-independent data exchange and thus common data models.

With domain-specific languages (domain specific languages, DSL) it is possible to describe precisely these data models and thus interfaces.

The individual products that are part of an autonomous system communicate not only their values ​​(e.g. sensor data) and instructions via these interfaces. The products also pass on information about themselves, e.g. their condition and the reliability or uncertainty of the communicated values.

Fraunhofer IESE has developed such languages ​​and associated tools (e.g. for MagicDraw and Enterprise Architect). The description language for reliability is based on a standard of the Object Management Group (OMG) and has been published open source.

Decision-making body

When all (safety-relevant) information about all products and the respective context is available, a (mostly central) decision-making body is required. These

  • checks whether the assumptions about the medical context are met,
  • checks how completely the technical requirements (which the manufacturer assumed at the time of development) are met,
  • anticipates the context-specific consequences of an action (e.g. administration of a drug through an infusion pump if one of the products is unsafe) and
  • executes the decisions and controls products, e.g. an infusion pump or an alarm.

This decision logic should also be formulated using domain-specific languages. This enables interoperability between products and manufacturers to be achieved.

Describing this “business logic” requires a high level of domain knowledge. Ultimately, this results in a formalization of medical knowledge.

Summary

The autonomous systems (or their decision-making bodies) decide on the behavior of the entire system. The system makes these decisions independently and context-specifically.

Domain-specific languages ​​describe both the context and the decision logic and thus

  • the products in the system,
  • their interfaces and
  • the effects of decisions in the event that the assumptions are met or that they are not met (e.g. products do not behave according to specifications).

4. Advantages of autonomous systems

a) Four advantages for medical device manufacturers

Greater patient safety

Manufacturers may and must specify the intended purpose and intended use and thus the clinical and technical context; however, they are hardly able to assess the risks that arise with all combinations of products and contexts.

This applies in particular to situations in which users do not use the products as intended.

This makes it difficult to control these risks and ensure patient safety.

Autonomous systems can (and should) check whether the manufacturer's assumptions are met at all times. Depending on the context, they then decide how security can be guaranteed even if the manufacturer's assumptions are not met.

No unnecessary product costs

Many manufacturers try to estimate the risks for the combinatorial variety of situations by making worst-case assumptions. In order to counter these, risk-minimizing measures are required on a regular basis. These lead to high costs or even to the fact that the product is no longer competitive and, as a result, can no longer be brought onto the market.

Autonomous systems should not assume a hypothetical worst-case scenario, but rather decide on the basis of the actual "case". Systems have more options than individual products for reacting to specific contexts.

This enables system manufacturers to implement not only more cost-effective (more efficient), but also more effective risk-minimizing measures through dynamic risk management. This in turn leads to greater patient safety.

More efficient development, documentation and approval

Most manufacturers try to use product variants and product families to meet the requirements of their customers on the one hand and to limit development costs on the other.

But this cost efficiency will not succeed if you have to create your own technical documentation for each product and variant and have to go through a separate approval.

In this apparent dilemma, model-based developments show their advantages:

  • Safety concepts only need to be modeled, developed and tested once. The modeling ensures that these concepts have been implemented identically and effectively for different variants and products. Variation points in the engineering artifact product line are connected to the safety artifact variation points and form a safe product line.
  • The modeling, e.g. with the help of domain-specific languages ​​or modeling tools, leads directly to the interface specification (and thus to product or component requirements). The models also form the consistent documentation at the same time, i.e. they are part of the technical documentation.
  • This also increases “regulatory compliance”. Because the frequently encountered and mostly incomplete “post-documentation” cannot happen with these model-based approaches.
  • In addition, the modeling allows the verification and validation of the product or component at an early stage (namely at the level of the model itself).

Conclusion: By modeling the products and the safety concepts, manufacturers save time and money for development and documentation. This advantage becomes more important the more frequently products or components are reused in different constellations, e.g. in product families.

Increasing the efficiency of other processes

This article only examines the autonomous systems in the context of medical devices. But manufacturers can also use autonomous systems in the production of medical products (comparable to the automotive industry) and in the automation of research (see a press release from Fraunhofer IESE).

b) Three advantages for operators (hospitals, hospitals, practices)

Legal compliance

Laws and ordinances such as the Medical Device Operator Ordinance place special demands on healthcare facilities that operate networked medical devices:

Medical devices connected to one another and medical devices connected to accessories including software or other objects may only be operated and used if they are suitable for use in this combination, taking into account the intended purpose and the safety of patients, users, employees or third parties.

MPBetreibV §4 (4)

But how should the operators meet these requirements? How are they supposed to be able to assess the risks if they only know about the products what is described in the accompanying materials?

Autonomous systems, the product interfaces of which are described using domain-specific languages, support the operator in fulfilling the requirements.

Now these prerequisites are not only formulated in the accompanying materials (if at all). Rather, the interface descriptions, which can also be evaluated by a computer, are a necessary prerequisite for using the product.

Well-founded selection of products

This allows healthcare facilities a more well-founded selection of products: only products that have these interface descriptions are even considered for use in autonomous systems.

Similar to the DICOM declarations of conformity (with IOD, SCP, SCU) guarantee the interoperability of DICOM-capable devices, products with the interfaces that are described via DSL allow the precise selection of interoperable products.

The operators can explicitly state these interoperability requirements in tenders. They can also select products more easily and replace them with other products as long as they have identical interfaces.

Greater safety for the patient

The most important advantage for the operator, however, is that the interoperability of the products and the ability of autonomous systems to react specifically to different contexts lead to greater patient safety.

Now this security no longer depends on the timely intervention of the medical staff and the specific medical and technical context. Because autonomous systems are characterized by the fact that they work independently and can adapt to different circumstances.

5. Autonomous systems from the perspective of medical device law

a) Medical device versus system

The term "system" associates systems made up of several products, e.g. in the sense of "systems and treatment units" according to Article 22 of the MDR. But this is not always the case.

Autonomous systems can be:

  1. A only medical device like an operating room robot. This would be a PEMS (Programmable Electrical Medical System) according to IEC 60601-1.
  2. One from a company Combination of products placed on the market from at least one medical device and, if applicable, from other products within the meaning of Article 22.
  3. Several products linked together by a healthcare facility (e.g. hospital) (including medical devices) that are supposed to achieve a specific purpose together.

MDR and IVDR only have specific requirements for the first two cases. In particular, the requirements for risk management are challenging: Manufacturers have to evaluate the combinatorial explosion of products, product states and contexts with and in which the products can be operated.

Standards such as IEC 80001-1 describe the risk management that healthcare facilities should operate that combine products into a "system".

b) Interoperability requirements

The intended use of an individual product can include being networked. The MDR therefore at least stipulates that manufacturers must guarantee the interoperability of their products. These requirements can be found, for example, in Appendix I.

Risks related to the possible negative interaction between software and the IT environment in which it is used and with which it interacts;

Products that are to be used together with other products or products that are not medical devices are designed and manufactured in such a way that the interaction and compatibility are reliable and safe.

Annex I, Section 14.2 and Section 14.5

c) classification

If a product contains a closed-loop system, manufacturers should study rule 22 of Annex VIII of the MDR.

Active therapeutic products with an integrated or built-in diagnostic function that significantly determines patient management by the product, such as closed control systems or automatic external defibrillators, belong to class III.

MDR, Annex VIII, Rule 22

However, this rule applies to individual products. The MDR does not have any specific rules for systems made up of several (medical) products.

d) Conclusion

Many regulations, e.g. the MDR, do not look at networked (autonomous) systems. They have requirements that (only) the individual products have to meet. These include the above-mentioned requirements for interoperability.

The MDR does not regulate the responsibility for the functioning of an (autonomous) system made up of several products from different manufacturers. This responsibility rests with the operators. But they are often overwhelmed with precisely this responsibility.

6. Summary

a) Autonomous systems combine many challenges

The advantages of autonomous systems are obvious. Many of the challenges too:

  • These are networked systems made up of many products. The combinatorial explosion of configurations and problems is difficult to control.
  • The products often use processes of the artificial intelligence. It is particularly difficult to prove the security and performance of these algorithms.
  • The autonomous systems are mostly closed-loop systems that are used in critical contexts. The Risks in these contexts are particularly high.

b) But there is no way around it

In the long run, the approach is too naive to want to treat patients individually, safely and effectively with isolated medical products and ignorance of specific technical and medical contexts.

The increasing shortage of medical personnel requires that systems actually react autonomously (i.e. without human intervention) and safely to contexts such as the failure of system components.

c) Is there a deficiency or incorrect regulation?

The EU regulations do not do justice to the current situation. They focus too much on the individual product and its interoperability. The system concept - and this does not mean systems within the meaning of Article 22 - is missing.

The possible joy of some manufacturers that something is not regulated may be premature. Because a lack of regulation leads to great uncertainty in the approval of the individual products. Maximum requirements from authorities and notified bodies and the discussion of unrealistic worst-case scenarios make approval more difficult.

The belief that the operators would effectively analyze and control the risks posed by the networked systems is nothing more than a belief.

d) There are great opportunities for good manufacturers

Manufacturers who are already failing to design medical devices and documentation in a modular way will find it difficult to develop products for use in autonomous systems.

Because modular architectures are the prerequisites for

  • the reusability, interchangeability and testability of components in different products and (autonomous) systems,
  • effective security concepts,
  • the expansion to include modular "safety artifacts" and
  • a lean, consistent and legally compliant documentation.

On the other hand, those manufacturers who have already designed model-driven modular architectures are well prepared. You will be rewarded with

  • a faster "time to market",
  • lower development and product costs,
  • a leaner documentation of your products and product families,
  • legal conformity that is easier to achieve and prove, and thus faster approval,
  • safer products and
  • greater competitiveness.

e) Take the first steps

Even the longest journey begins with the first step. Examples of successful first steps are:

  • Define clinical contexts that manufacturers want to serve with their own products
  • Check the modularity of your own products
  • Determine the state of modeling in your own company
  • Offer the developers further training on the following topics:
    • Autonomous systems
    • Modeling
    • Domain-specific questions
    • Security architectures
    • Risk management
    • Regulatory requirements
  • Discuss a small feasibility study with experts in autonomous systems

Similar to the use of computer-based modeling and simulation, the hurdle for developing autonomous systems is high. Without a clear strategy and long-term investments in employees, concepts and tools, the development of autonomous systems will not be crowned with success either.

On the other hand, those companies will gain decisive competitive advantages that consistently continue on this path.


For further questions, please contact Dr. Rasmus Adler from Fraunhofer IESE and the whole team from the Johner Institute, e.g. via the free micro-consulting.