Quality attributes are not visibly listed in requirement documentation. They are hidden between the lines and difficult to decipher. Often time stakeholders have little idea about how a QA would affect the software unless an Architect helps them to see connection between requirement and QA. Architect has to bring QA forward and list them out. This helps an architect to focus on items which are keys to developing successful software architecture under given constraints.
Architecturally Significant Requirements (ASR)
Refer Requirement discovery and constraint Analysis under Business Technology Strategy for how to do requirement discovery.
The first step in collecting ASR is to read the requirement documents carefully. Look for requirements that fall in one of the following categories:
- Allocation of responsibilities:
- Find out how the responsibilities are allocated to various elements of software.
- Coordination Model
- List out elements that must or must not coordinate
- Determine properties of the coordination
- Choose the communication method within or without the system.
- Data Model
- How the data of the system should be created, stored, accessed, changed, destroyed etc.?
- Management of Resources
- Decisions which affect resource management
- Mapping among architectural elements
- Locate any identified mapping structure defined. For e.g. mapping of modules to runtime elements, runtime elements to processors etc
- Choice of Technology
- Are there any technologies identified to be used? Is there a particular environment the system has to run?
- Binding Time decisions
- This covers every other category listed above. In each case there is a binding time decision involved.
- For e.g. in resource management, the system should be able to accept a new device that is plugged in at runtime.
Quality Attribute Workshop (QAW)
The next step is to interview stakeholders. If the number of stakeholders is large one has to choose whom to talk to. This could be dependent on the size of the effect on the design of the system by stakeholders.
QAW is a method for talking to stakeholders in a pre-defined setting. The steps involved in QAW are as follows:
Step 1: QAW Presentation and Introduction.
This is done by QAW facilitator. QAW facilitator is mostly the software architect.
Step 2: Business Presentation
This is done by Management Representative. Aim is to explain the system’s business context.
Step 3: Architectural Plan Presentation
This is done by architect. Architect provides a rough system architectural plan as it stands at that time
Step 4: Identification of Architectural Drivers.
Selection of drivers based on conversation in step 2 and 3 above. Ask the participants to clarify, add, and delete the drivers.
Step 5: Scenario Brainstorming
Architect should ensure that there is at least one Scenario for each driver in step 4.
Step 6: Scenario Consolidation
Stakeholders push backs are normal on non-functional requirements. An architect has to demonstrate how a QA is related to the requirement to avoid such push backs. The stakeholders are not interested in spending on requirements that are not listed by them.
It is necessary to identify an ASR and the related QA to answer queries on non-functional requirements. Cost-benefit of a QA and its role in architecture should be described in a layman’s words to get stakeholders buy-in.
Stakeholders involve architect at an early stage to get the architecture correct from the current and future requirements perspective.
Stakeholders should understand that same requirement can have very different implementation from different people. Success depends on how well the ASR is identified and mapped to one or more QA. Since stakeholders are involved in for a long haul, they want continuity with the software in future. For the sake of using software for a long time it is necessary to involve an Architect early on because architecture cannot be changed at a later date and rewrite is very costly.
Quality Attribute Scenario:
As seen above the writing of an appropriate QA Scenario is very important to get the ASR right. Quality attribute scenario has the following important parts:
- Who is Stimulating?
- It could be a human, a computer, fault in the system etc
- What it does to the system?
- It could be an event that occurs, an attack in progress etc
- It could be a user clicking on the menu, typing in the text box etc
- What response to give to the stimulus received?
- How to measure the response?
- Is it satisfactory to prevent the attack? Or, additionally, inform the people watching the system and log the information too?
- In case of performance, is it sufficient to measure the latency? Or, need to look into the processing time too.
- What environment this stimulus occurs in?
- Do we have control over the environment? Can we change it for the better?
- If not, do we have to prepare our system to handle such issues better next time?
- Is it a specific component of the system that is affected or a sub-system?
Two characteristic comes if QA is written this way. One, since specific measurement of the response has to be documented, the scenario becomes testable. Second, since we know clearly the stimulus, response, environment it occurs in and the artifacts it affects, it is unambiguous.
Quality Attribute Testing
The testing of QA differs from one attribute to another. Some of them can be tested using Analytical model others need a checklist. In some cases one need to create model (prototype) to test. We will cover some of the QA here.
Assume the software has used Model View Controller (MVC) design. We need to know the following information
- Arrival rate of events
- Selected Queue discipline
- Selected scheduling algorithm
- Service time for events
- Network topology
- What time it takes to process the message in view/model/controller
- Bandwidth of the network connect to view/model/controller
With these details one can calculate the latency of the system.
In case of security mostly it is a checklist. The checklist is sourced from many sources. For e.g. for payment receiving websites it has to follow the checklist of Payment Card Industry (PCI) as part of their security checklist.
The checklist can also be sourced from internal to the company. For e.g. if company do not serve a particular country it may not permit log-in from that country. The security testing checklist should include a test around who can or cannot log-in including successful and failure tests. This is the access rights testing.
Balancing and optimizing quality attributes
Refer Balancing and Optimizing Quality attributes under Quality Attribute Pillar
The most common evaluation method is Architecture Trade-off Analysis Method(ATAM).
It has 4 phases as follows:
- Partnership and preparation. Here the evaluation team leadership meet to list out the details for ATAM exercise. Information gathering for the exercise is done here.
- (evaluation phase 1) This is a continuation of the phase one with more analysis.
- Details on business drivers
- During QAW the architecture was not detailed enough. By now the architect will be able to cover all necessary parts in enough details for the participants to evaluate the system.
- List out architectural approaches.
- Generate Quality Attribute Utility tree.
- Analyze approaches taken
- Some more evaluation (evaluation phase 2)
- There is a gap between the first evaluation and this one.
- A summary of the first evaluation is created.
- More scenarios are analyzed if required.
- Brainstorming on scenarios takes place. Scenarios are prioritized.
- Architectural approaches are analyzed.
- Present the result of the evaluation to the stakeholders
- Follow-up. Prepare final report and submit.
Economic analysis of Quality Attributes
Also refer Business Valuation under Business Technology Strategy
When listing out various design strategies for a Quality Attribute the architect should be able to weigh the financial cost of the strategy too.
This starts with the QA Scenarios collected and refined in QAW and similar efforts. Then effort was put in to map architectural strategies to responses in scenarios. There could be some side-effects on other QAs. The side-effects could be negative. This leads to capture the utility of alternative responses in a scenario in a utility-response curve. With this curve it becomes easy to calculate the total benefit for a strategy. Estimate the cost of implementing the strategy. The ratio of total benefit and cost of implementing will give the Value of cost. Use the ratio value to select the strategy to be implemented.
The process explained above is called Cost Benefit Analysis Method (CBAM).