ISTQB CT-AI Certified Tester AI Testing Exam Exam Practice Test

Recognizing digits from a scan of handwritten numbers using machine learning is an example of classification. Here’s a breakdown:

Classification: This type of machine learning involves categorizing input data into predefined classes. In this scenario, the input data (handwritten digits) are classified into one of the 10 digit classes (0-9).

Why Not Other Options:

Reinforcement Learning: This involves learning by interacting with an environment to achieve a goal, which does not fit the problem of recognizing digits.

Regression: This is used for predicting continuous values, not discrete categories like digit recognition.

Clustering: This involves grouping similar data points together without predefined classes, which is not the case here.

References: The explanation is based on the definitions of different machine learning types as outlined in the ISTQB CT-AI syllabus, specifically under supervised learning and classification​​.

Clustering is a type of machine learning technique used to group similar data points into clusters. It is a key concept in unsupervised learning, where the algorithm tries to find patterns or groupings in data without prior knowledge of output classes. Let's analyze each option:

A. Clustering is classification of a continuous quantity.

This is incorrect. Classification typically involves discrete categories, whereas clustering involves grouping similar data points. Classification of continuous quantities is generally referred to as regression.

B. Clustering is supervised learning.

This is incorrect. Clustering is an unsupervised learning technique because it does not rely on labeled data.

C. Clustering is done without prior knowledge of output classes.

This is correct. In clustering, the algorithm groups data points into clusters without any prior knowledge of the classes. It discovers the inherent structure in the data.

D. Clustering requires you to know the classes.

This is incorrect. Clustering does not require prior knowledge of classes. Instead, it aims to identify and form the classes or groups based on the data itself.

Therefore, the correct answer is C because clustering is an unsupervised learning technique done without prior knowledge of output classes​​.

 Challenges for Acquiring Test Data in ML Systems: Compliance needs, the changing nature of data over time, and sourcing data from public sources are significant challenges. Data being generated quickly is generally not a challenge; it can actually be beneficial as it provides more data for training and testing.

 Reference: ISTQB_CT-AI_Syllabus_v1.0, Sections on Data Preparation and Data Quality Issues​​​​.

 Technology Most Typically Used to Implement AI: Genetic algorithms are a well-known technique used in AI. They are inspired by the process of natural selection and are used to find approximate solutions to optimization and search problems. Unlike search engines, procedural programming, or case control structures, genetic algorithms are specifically designed for evolving solutions and are commonly employed in AI implementations.

 Reference: ISTQB_CT-AI_Syllabus_v1.0, Section 1.4 AI Technologies, which identifies different technologies used to implement AI​​.

A. Identifying suitable tests by looking at the complexity of the test cases.

While complexity analysis can help in selecting important test cases, it does not directly address the issue of optimizing the entire regression suite effectively.

B. Using a random subset of tests.

Randomly selecting test cases may miss critical tests and does not ensure an optimized regression suite. This approach lacks a systematic method for ensuring comprehensive coverage.

C. Automating test scripts using AI-based test automation tools.

Automation helps in running tests efficiently but does not inherently optimize the selection of tests to prevent the suite from growing too large.

D. Using an AI-based tool to optimize the regression test suite by analyzing past test results.

This is the most effective approach as AI-based tools can analyze historical test data, identify patterns, and prioritize tests that are more likely to catch defects based on past results. This method ensures an optimized and manageable regression test suite by focusing on the most impactful test cases.

Therefore, the correct answer is D because using an AI-based tool to analyze past test results is the best option to optimize regression test selection and manage the size of the regression suite effectively​​.

The question asks which test is least likely to be performed during the ML model testing phase. Let's consider each option:

Testing the accuracy of the classification model (A): Accuracy testing is a fundamental part of the ML model testing phase. It ensures that the model correctly classifies the data as intended and meets the required performance metrics.

Testing the API of the service powered by the ML model (B): Testing the API is crucial, especially if the ML model is deployed as part of a service. This ensures that the service integrates well with other systems and that the API performs as expected.

Testing the speed of the training of the model (C): This is least likely to be part of the ML model testing phase. The speed of training is more relevant during the development phase when optimizing and tuning the model. During testing, the focus is more on the model's performance and behavior rather than how quickly it was trained.

Testing the speed of the prediction by the model (D): Testing the speed of prediction is important to ensure that the model meets performance requirements in a production environment, especially for real-time applications.

References:

ISTQB CT-AI Syllabus Section 3.2 on ML Workflow and Section 5 on ML Functional Performance Metrics discuss the focus of testing during the model testing phase, which includes accuracy and prediction speed but not the training speed.

For AI-enabled medical devices, the most required aspects for certification are safety, transparency, and side effects. Here’s why:

Safety (Aspect III): Critical for ensuring that the AI system does not cause harm to patients.

Transparency (Aspect IV): Important for understanding and verifying the decisions made by the AI system.

Side Effects (Aspect V): Necessary to identify and mitigate any unintended consequences of the AI system.

Why Not Other Options:

Autonomy and Maintainability (Aspects I and II): While important, they are secondary to the immediate concerns of safety, transparency, and managing side effects in life-critical processes.

References: This explanation is aligned with the critical quality characteristics for AI-based systems as mentioned in the ISTQB CT-AI syllabus, focusing on the certification of medical devices​​.

Pairwise testing is used to handle the large number of combinations of parameters that can arise in complex systems like self-driving cars. The question asks which of the given options is least likely to be a reason for the explosion in the number of parameters.

Different Road Types (A): Self-driving cars must operate on various road types, such as highways, city streets, rural roads, etc. Each road type can have different characteristics, requiring the car's system to adapt and handle different scenarios. Thus, this is a significant factor contributing to the growth of parameters.

Different Weather Conditions (B): Weather conditions such as rain, snow, fog, and bright sunlight significantly affect the performance of self-driving cars. The car's sensors and algorithms must adapt to these varying conditions, which adds to the number of parameters that need to be considered.

ML Model Metrics to Evaluate Functional Performance (C): While evaluating machine learning (ML) model performance is crucial, it does not directly contribute to the explosion of parameter combinations in the same way that road types, weather conditions, and car features do. Metrics are used to measure and assess performance but are not themselves variable conditions that the system must handle.

Different Features like ADAS, Lane Change Assistance, etc. (D): Advanced Driver Assistance Systems (ADAS) and other features add complexity to self-driving cars. Each feature can have multiple settings and operational modes, contributing to the overall number of parameters.

Hence, the least likely reason for the incredible growth in the number of parameters is C. ML model metrics to evaluate the functional performance.

References:

ISTQB CT-AI Syllabus Section 9.2 on Pairwise Testing discusses the application of this technique to manage the combinations of different variables in AI-based systems, including those used in self-driving cars.

Sample Exam Questions document, Question #29 provides context for the explosion in parameter combinations in self-driving cars and highlights the use of pairwise testing as a method to manage this complexity.

 AI-Based Autonomous Functions: An AI-based autonomous system is one that can respond to its environment without human intervention. The other options either involve human decisions or do not use AI at all.

 Reference: ISTQB_CT-AI_Syllabus_v1.0, Sections on Autonomy and Testing Autonomous AI-Based Systems​​​​.

The question refers to a problem where data used for an object detection ML system was labelled incorrectly. This issue is most closely related to "accuracy issues." Here's a detailed explanation:

Accuracy Issues: The primary goal of labeling data in machine learning is to ensure that the model can accurately learn and make predictions based on the given labels. Incorrectly labeled data directly impacts the model's accuracy, leading to poor performance because the model learns incorrect patterns.

Why Not Other Options:

Security Issues: This pertains to data breaches or unauthorized access, which is not relevant to the problem of incorrect data labeling.

Privacy Issues: This concerns the protection of personal data and is not related to the accuracy of data labeling.

Bias Issues: While bias in data can affect model performance, it specifically refers to systematic errors or prejudices in the data rather than outright incorrect labeling.

References: This explanation is consistent with the concepts covered in the ISTQB CT-AI syllabus under dataset quality issues and their impact on machine learning models​​.

The question asks which characteristic is least likely to cause safety-related issues for an AI system. Let's evaluate each option:

Non-determinism (A): Non-deterministic systems can produce different outcomes even with the same inputs, which can lead to unpredictable behavior and potential safety issues.

Robustness (B): Robustness refers to the ability of the system to handle errors, anomalies, and unexpected inputs gracefully. A robust system is less likely to cause safety issues because it can maintain functionality under varied conditions.

High complexity (C): High complexity in AI systems can lead to difficulties in understanding, predicting, and managing the system's behavior, which can cause safety-related issues.

Self-learning (D): Self-learning systems adapt based on new data, which can lead to unexpected changes in behavior. If not properly monitored and controlled, this can result in safety issues.

References:

ISTQB CT-AI Syllabus Section 2.8 on Safety and AI discusses various factors affecting the safety of AI systems, emphasizing the importance of robustness in maintaining safe operation.

Flexibility in AI systems is crucial for various reasons, particularly because it allows for easier modification and adaptation of the system as a whole.

AI systems are inherently flexible (A): This statement is not correct. While some AI systems may be designed to be flexible, they are not inherently flexible by nature. Flexibility depends on the system's design and implementation.

AI systems require changing operational environments; therefore, flexibility is required (B): While it's true that AI systems may need to operate in changing environments, this statement does not directly address the importance of flexibility for the modification of the system.

Flexible AI systems allow for easier modification of the system as a whole (C): This statement correctly describes the importance of flexibility. Being able to modify AI systems easily is critical for their maintenance, adaptation to new requirements, and improvement.

Self-learning systems are expected to deal with new situations without explicitly having to program for it (D): This statement relates to the adaptability of self-learning systems rather than their overall flexibility for modification.

Hence, the correct answer is C. Flexible AI systems allow for easier modification of the system as a whole.

References:

ISTQB CT-AI Syllabus Section 2.1 on Flexibility and Adaptability discusses the importance of flexibility in AI systems and how it enables easier modification and adaptability to new situations.

Sample Exam Questions document, Question #30 highlights the importance of flexibility in AI systems.