Model-Based Dependability Evaluation of Complex Critical Control Systems

Computer systems used in critical control applications are rapidly growing in complexity, featuring a very high number of requirements together with large, distributed and heterogeneous architectures, both at the hardware and software levels. Their dependability requirements are even more demanding, calling for more detailed analyses in order for the system to be evaluated against them. Complex systems are usually evaluated against safety requirements using simulation based techniques, while formal methods are used to verify only limited parts of the system. When evaluating system as a whole (i.e. as a “black-box”), the simulation techniques belong to the class of functional testing. Traditional functional testing techniques reveal inadequate for the verification of modern control systems, for their increased complexity and criticality properties. Such inadequateness is the result of two factors: the first is specification inconsistency (impacting on testing effectiveness), the second is test-case number explosion (impacting on testing efficiency). In particular, as the complexity of the system grows up, it is very difficult (nearly impossible) to develop a stable system requirements specification. This is a traditional problem, which is even more critical with nowadays control systems, featuring thousands of functional requirements and articulated architectures. Accurately revising natural language specification only reduces the problem. Missing requirements can be added at any time, others are continuously modified; negative requirements are usually stably missing. Another problem which is not addressed by traditional techniques is the impossibility to bias the test set in order to balance test effectiveness (number of potentially discovered errors) and efficiency (time required for test execution). Grey-box approaches allow to measure test-effectiveness by integrating the code coverage measurement; however, a stronger integration between static and dynamic analyses techniques is needed to enhance functional testing. To cope with the aforementioned issues, in this thesis we present a hybrid and grey-box functional testing technique based on influence variables and reference models (e.g. UML class and state diagrams) which is aimed at making functional testing of critical systems both feasible and cost-effective. While component or even subsystem level reliability can be evaluated quite easily, when availability has to be evaluated against system level requirements for non trivial failure modes, together with maintainability constraints, no traditional technique (e.g. Faul Trees) seems to be adequate. To cope with such issue, designers use to specify more conservative constituent level requirements, ensuring feasibility at a higher expense. However, more strict constituent level requirements feature two disadvantages: as first, system level availability target is not guaranteed to be fulfilled, as it depends on non straightforward architectural dependencies (i.e. how constituent are connected and interact with each other); secondly, even though more strict constituent level reliability requirements would ensure the accomplishment of global availability goals, this would imply a non balanced allocation of costs, as developers would not be able to fine tune system reliability parameters. The impossibility to correctly size reliability parameters according to structural dependencies is very likely to take to over-dimension some components and under-dimension some others, with a negative cost impact. A combination of techniques is therefore needed to perform system level availability modeling of complex heterogeneous control systems, considering both structural and behavioral studies. While a single highly expressive formalism (e.g. high-level Petri Nets) could be adopted to model the entire system from an availability point of view, there are at least two issues which are related to such choice: the first is efficiency, as the more expressive formalism usually features more complex solving algorithms; the second is ease of use, as the more expressive formalisms usually require a skilled modeler. The former issue is such to impede the application of an entire class of formalisms to large systems (e.g. for the state-space explosion problem). The explicit use of more formalisms allow modelers to fine tune the choice of the formalism to the needed modeling power. Furthermore, implicit multiformalism allows modelers to specify new formalisms out of existing ones, thus allowing to combine enhanced power, better efficiency and increased ease of use (the complexity of the solving process is hidden to the modeler). In this thesis we show how multiformalism approaches apply to critical control systems for system availability evaluation purposes.