Databricks Databricks-Machine-Learning-Professional Databricks Certified Machine Learning Professional Exam Practice Test

The python_function model flavor is a generic model interface for MLflow Python models. It ensures that any MLflow Python model can be loaded and interacted with using a consistent API, regardless of which library was used to create the model. The python_function model flavor defines a standard format for model data and a common interface for model inference. The model data consists of a predict function that can accept various types of input data and return various types of output data. The model inference interface consists of a load_pyfunc function that can load the model from a given path and return an object with a predict method. The python_function model flavor enables model portability and interoperability across different platforms and environments, as it allows deployment tools to understand and use the model without having to integrate with each library-specific model flavor12

The other options are incorrect because:

• A. python_function provides several benefits over the built-in library-specific model flavors, such as portability, interoperability, and simplicity.
• B. python_function does not directly enable parallelizable deployment of models, as it depends on the underlying implementation of the predict function and the deployment tool. However, python_function can be used with other tools and frameworks that support parallelizable deployment, such as Spark, Databricks, or Ray.
• D. python_function is not used to store models in an MLmodel file, but rather to load models from an MLmodel file. The MLmodel file is a configuration file that contains metadata about the model, such as the model flavor, the data path, the dependencies, etc. The MLmodel file is created when the model is logged or saved using MLflow3
• E. python_function does not directly enable deployment of models in batch, streaming, or real-time environments, as it depends on the underlying implementation of the predict function and the deployment tool. However, python_function can be used with other tools and frameworks that support different deployment scenarios, such as MLflow Serving, MLflow Projects, MLflow Models, or MLflow Model Registry4

References:

• mlflow.pyfunc — MLflow 2.9.1 documentation
• Models, Flavors, and PyFuncs in MLflow
• MLflow Models — MLflow 2.9.1 documentation
• Built-In Deployment Tools - MLflow 2.9.1 documentation

MLflow Model flavors are a key concept that makes MLflow Models powerful: they are a convention that deployment tools can use to understand the model, which makes it possible to write tools that work with models from any ML library without having to integrate each tool with each library. MLflow defines a set of built-in model flavors for popular ML frameworks, such as scikit-learn, TensorFlow, PyTorch, etc. Each flavor defines a standard format for model data and a common interface for model inference. For example, the python_function flavor allows any model to be loaded as a Pythonfunction that can be applied to input data. MLflow also supports custom model flavors for user-defined models or community-developed flavors. MLflow Model flavors enable model portability and interoperability across different platforms and environments12 References:

• MLflow Models — MLflow 2.9.1 documentation
• Models, Flavors, and PyFuncs in MLflow

Real-time deployment is a paradigm that can centrally compute predictions for a single record with exceedingly fast results. Real-time deployment involves exposing a model as a REST API endpoint that can receive requests and return predictions in milliseconds. This paradigm is suitable for scenarios where low latency and high throughput are required, such as fraud detection, recommendation systems, or chatbots. The other deployment paradigms are not as fast or as centralized as real-time deployment. Streaming deployment involves processing data streams in micro-batches, which can introduce some delay. Batch deployment involves processing large batches of data at scheduled intervals, which can take hours or days. Edge or on-device deployment involves deploying models to edge devices, such as mobile phones or IoT sensors, which can reduce the network overhead but also limit the computational power and model size. https://docs.databricks.com/en/machine-learning/ml-tutorials.html

Feature drift is a change in the distribution of the input data over time, which can affect the performance and accuracy of the model. Monitoring feature drift is important to ensure that the model is still valid and reliable for the current data. One simple and low-cost method of monitoring numeric feature drift is to track the summary statistics trends of the input features, such as mean, standard deviation, minimum, maximum, etc. These statistics can be computed easily and efficiently, and can provide a quick overview of the changes in the data distribution. If the summary statistics deviate significantly from the baseline values, it may indicate a feature drift. However, summary statistics trends may not capture all the nuances of the data distribution, such as outliers, multimodality, skewness, etc. Therefore, other methods, such as statistical tests or visualizations, may be needed to complement the summary statistics trends and provide a more comprehensive analysis of the feature drift123 References:

• Monitoring Feature Drift - Databricks
• Monitor feature attribution skew and drift | Vertex AI | Google Cloud
• Monitoring Model Drift - 6 Different Methodologies - Qualdo

The code block that the team is using is correct and does not need any changes to accomplish the task. The update_registered_model method of the MlflowClient class can be used to update the metadata of a registered model, such as its name or description. The method takes the following parameters:

• name: The name of the registered model to update.
• description: The new description for the registered model.
• new_name: The new name for the registered model.

The method returns a RegisteredModel object that represents the updated registered model1

The other options are incorrect because:

• A. Replacing update_registered_model with update_model_version would not update the metadata of the registered model, but rather the metadata of a specific model version. The update_model_version method can be used to update the stage, description, or name of a model version. The method takes the following parameters:

The method returns a ModelVersion object that represents the updated model version2

• C. Replacing description with artifact would not update the description of the registered model, but rather raise an error, as artifact is not a valid parameter for the update_registered_model method. The artifact parameter is used for the log_model or save_model methods, which can be used to log or save a model with a specific flavor and artifact path34
• D. Replacing client.update_registered_model with mlflow would not update the registered model, but rather raise an error, as mlflow is not a valid method, but rather a module that contains various submodules and functions. The mlflow module does not have an update_registered_model function, but rather the MlflowClient class has an update_registered_model method5
• E. Adding a Python model as an argument to update_registered_model would not update the registered model, but rather raise an error, as a Python model is not a valid argument for the update_registered_model method. The update_registered_model method does not take a model as an argument, but rather the name and description of the registered model to update. To add a Python model to the Model Registry, the create_model_version method of the MlflowClient class can be used, which takes the following parameters:

The method returns a ModelVersion object that represents the created model version6

References:

• mlflow.tracking.client.MlflowClient.update_registered_model — MLflow 2.9.1 documentation
• mlflow.tracking.client.MlflowClient.update_model_version — MLflow 2.9.1 documentation
• mlflow..log_model — MLflow 2.9.1 documentation
• mlflow..save_model — MLflow 2.9.1 documentation
• mlflow — MLflow 2.9.1 documentation
• mlflow.tracking.client.MlflowClient.create_model_version — MLflow 2.9.1 documentation

MLflow Model Serving allows you to deploy MLflow models for real-time serving via REST API. When you enable Model Serving for a registered model, all model versions in the Staging and Production stages are automatically deployed. You can also manually deploy model versions in other stages, such as None or Archived, by using the UI or the API. However, only the model versions in the Staging and Production stages are automatically deployed by default. References: MLflow Model Serving, Deploy MLflow models

 In a continuous integration, continuous deployment (CI/CD) process for machine learning pipelines, the arrival of a new model version in the MLflow Model Registry is a common event that triggers the execution of automated testing. This is because the MLflow Model Registry is a centralized repository for managing the full lifecycle of MLflow models. It provides model lineage, model versioning, stage transitions, and annotations. When a new model version is registered, it can be automatically tested for quality, performance, and compliance using various tools and frameworks, such as MLflow Projects, MLflow Models, MLflow Tracking, and MLflow Model Serving. The automated testing can help ensure that the new model version meets the desired criteria and standards before it is deployed to production or other stages. The automated testing can also help detect and prevent any errors, bugs, or drifts that may affect the model functionality or accuracy. References:

• MLflow Model Registry Documentation, p. 1-2
• MLOps: Continuous delivery and automation pipelines in machine learning, Continuous integration and delivery (CI/CD) for ML, p. 5-6
• Machine Learning Engineering with MLflow, Testing and Validating ML Models, p. 1-2
• CI/CD for Machine Learning: What it is & Benefits in 2023 - AIMultiple, CI/CD for Machine Learning, p. 1-2

To view the data visualizations that are logged with each run, you can go to the Artifacts section of the MLflow Run page in Databricks. The Artifacts section shows the files and directories that are logged as artifacts for a run. You can browse the artifact hierarchy and preview the files, such as images, text, or HTML1. You can also download the artifacts or copy their URIs for further use2. The other options are incorrect because:

• Option A: The MLflow Model Registry Model page shows the information and metadata of a registered model, such as its name, description, versions, stages, and lineage. It does not show the data visualizations that are logged with each run3.
• Option B: The Artifacts section of the MLflow Experiment page shows the artifacts that are logged for an experiment, not for a specific run. It does not allow you to preview the files or browse the artifact hierarchy4.
• Option C: Logged data visualizations can be viewed in Databricks using the Artifacts section of the MLflow Run page1.
• Option E: There is no Figures section of the MLflow Run page in Databricks. The Figures section is only available in the open source MLflow UI, which shows the plots that are logged as figures for a run5. References: View run artifacts, Log, list, and download artifacts, Manage models, View experiment artifacts, Logging Visualizations with MLflow

The log_metric function is used to log a numeric metric value to the current MLflow run. It takes the name of the metric and the value as arguments. In this case, the name of the metric is “rmse” and the value is the variable rmse. The log_metric function can be used multiple times to log different metrics or update the same metric. The log_artifact function is used to log a file or directory as an artifact to the current MLflow run. The log_model function is used to log a model with a specified flavor to the current MLflow run. The log_param function is used to log a key-value pair as a parameter to the current MLflow run. There is a way to store values like this using the log_metric function, so option E is incorrect.https://docs.databricks.com/

The DESCRIBE HISTORY command can be used to view the commit history of a Delta table, including the schema changes, operations, and timestamps. This command can help identify when a column was dropped from the feature set and by which operation. The other commands are either invalid or do not provide the required information. References:

• Delta Lake - View Commit History
• Databricks Certified Machine Learning Professional Exam Guide - Section 1: Experimentation - Data Management

The MLflow Model Registry allows users to manage the full lifecycle of MLflow Models. It provides model lineage (which MLflow Experiment and Run produced the model), model versioning, stage transitions (for example from staging to production), and annotations. Users can use the MLflow Client API to interact with the Model Registry programmatically.

To move a model version from one stage to another, users can use the transition_model_version_stage method of the MLflow Client. This method takes the following parameters:

• name: The name of the registered model.
• version: The version number of the model.
• stage: The name of the stage to transition the model version to.
• archive_existing_versions: (Optional) If True, archive all other model versions in the same stage.

Therefore, to move the model version model_version for the model model from the Staging stage to the Production stage, and archive any existing model versions in the Production stage, the correct code block is:

PythonAI-generated code. Review and use carefully. More info on FAQ.

client.transition_model_version_stage(

name=model,

version=model_version,

stage="Production",

archive_existing_versions=True

)

References:

• [MLflow Model Registry - Databricks]
• [MLflow Client API - Databricks]
• [MLflow Model Registry API - Databricks] Message has links.

Drift is the change over time in the statistical properties of the data that was used to train a machine learning model. This can cause the model to become less accurate or perform differently than it was designed to1. Drift can be detected by monitoring the statistics of the input and output data over time and comparing them with the baseline statistics from the training data2. Depending on the type and severity of the drift, different responses may be appropriate. Some possible responses are:

• Retraining and deploying a model on more recent data: This can help the model adapt to the changes in the data and improve its performance. However, this may require frequent retraining and deployment cycles, which can be costly and time-consuming. Also, retraining may not be sufficient if the drift is caused by a change in the underlying concept or relationship between the input and output variables3.
• Rebuilding the machine learning application with a new label variable: This can help the model capture the new concept or relationship that has emerged in the data. However, this may require a significant redesign of the application and the data pipeline, as well as collecting and labeling new data. Also, rebuilding may not be feasible if the concept or relationship is constantly changing or unknown3.
• Sunsetting the machine learning application: This can help avoid the risks and costs of maintaining a model that is no longer reliable or useful. However, this may mean losing the benefits and value of the application and the data. Also, sunsetting may not be an option if the application is critical or mandatory for the business or the users3.

Therefore, all of these responses are probable, depending on the situation and the trade-offs involved. References:

• Databricks Machine Learning Professional Exam Guide, Section 4: Solution and Data Monitoring, p. 5
• Databricks Machine Learning Documentation, Monitoring ML Models, Data Drift Detection, p. 2-3
• A Gentle Introduction to Concept Drift in Machine Learning, Types of Concept Drift, p. 3-4
• Understanding Data Drift and Model Drift: Drift Detection in Python, Types of Drift, p. 2-3

Batch serving is a type of model deployment where predictions are computed in advance and stored for later use. This can be useful when the feature values are known ahead of time and the predictions need to be delivered quickly. Querying stored predictions can be faster than computing predictions in real-time, especially if the model is complex or the data volume is high. Real-time serving is a type of model deployment where predictions are computed on-demand at query time. This can be useful when the feature values are not known in advance or the predictions need to be up-to-date. However, computing predictions in real-time can be slower than querying stored predictions, especially if the model is simple or the data volume is low. References: [Batch Serving], [Real-time Serving] Message has links.

To restore the original scikit-learn model object, the correct method is mlflow.sklearn.load_model, which takes the model URI as an argument and returns a scikit-learn model. This method can load models with the sklearn flavor, which preserves the original attributes and methods of the model. The mlflow.load_model method is a generic method that can load models with any flavor, but it returns a pyfunc model, which does not have the feature_importances_ attribute. The client.list_artifacts method is used to list theartifacts associated with a run, not to load a model. The client.pyfunc.load_model method is similar to mlflow.load_model, but it takes a local file path instead of a model URI. The MLflow Experiments UI can show the logged parameters, metrics, and artifacts of a run, but it cannot show the model object itself. References:

• MLflow Models
• MLflow Python API

To log feature importance data from a CSV file as an artifact of an MLflow run, the correct method is mlflow.log_artifact, which takes the local file path and the artifact path as arguments.The mlflow.log_model method is used to log a model with a specific flavor, such as pyfunc or spark. The mlflow.log_data method does not exist. The mlflow.log_model_and_data method also does not exist, and it is not necessary to log both the model and the data in this case. References:

• MLflow Tracking API
• MLflow Models

MLflow Model Serving is a feature that allows users to host MLflow models as REST endpoints on Databricks1. To query a model version that is in the Production stage of the MLflow Model Registry, the user can use the following model URI format: https:// /model-serving///invocations2. In this case, the model name is recommender and the stage is Production, so the correct model URI is https:// /model-serving/recommender/Production/invocations. The other options are incorrect because they either use the wrong prefix (model instead of model-serving), the wrong stage name (stage-production instead of Production), or require the version number of the model version in Production, which is not necessary for querying the model by stage. References: Announcing MLflow Model Serving on Databricks, [MLflow Model Serving], [MLflow Model Registry]

 To read data from a stream source, such as Kafka, socket, or rate, the spark.readStream method should be used instead of spark.read. The spark.readStream method returns a streaming DataFrame that represents the unbounded input data stream. The spark.readStream method supports the same options and formats as the spark.read method, such as schema, delta, csv, json, etc. The spark.readStream method can also read from a Delta table as a stream source, by specifying the format("delta") and the path or table name of the Delta table123

The other options are incorrect because:

• A. Replacing “inference” with the path to the location of the Delta table does not change the fact that spark.read is used to read from a stream source, which is not supported. The spark.readStream method should be used instead, and the path or table name of the Delta table can be specified as an option or argument.
• B. Replacing schema(schema) with option("maxFilesPerTrigger", 1) does not change the fact that spark.read is used to read from a stream source, which is not supported. The spark.readStream method should be used instead, and the schema can be specified as an option or argument. The option("maxFilesPerTrigger", 1) is an optional configuration that limits the number of files processed in each trigger for file-based stream sources, such as delta, csv, json, etc. It does not affect the reading of data from a stream source4
• D. Replacing format("delta") with format("stream") does not change the fact that spark.read is used to read from a stream source, which is not supported. The spark.readStream method should be used instead, and the format can be specified as an option or argument. The format("stream") is not a valid format for reading data from a stream source. The supported formats are delta, kafka, socket, rate, etc1
• E. Replacing predict with a stream-friendly prediction function does not change the fact that spark.read is used to read from a stream source, which is not supported. The spark.readStream method should be used instead, and the prediction function can be applied to the streaming DataFrame as usual. The predict function does not need to be changed, as long as it can accept a streaming DataFrame as input and return a column of predictions as output5

References:

• Input Sources - Structured Streaming Programming Guide - Spark 3.2.0 Documentation
• Structured Streaming + Delta Lake - Databricks
• Structured Streaming Programming Guide - Spark 3.2.0 Documentation
• Configuration - Structured Streaming Programming Guide - Spark 3.2.0 Documentation
• Machine Learning with Structured Streaming - Databricks

MLflow provides a module called mlflow.shap that can be used to automatically calculate and log a Shapley feature importance plot using the SHAP library. The mlflow.shap.log_explanation function takes a predict function and a sample of input data, and computes the SHAP values and base values for each feature. It then logs the following artifacts to the current run:

• Base values
• SHAP values (computed using shap.KernelExplainer)
• Summary bar plot (shows the average impact of each feature on model output) The mlflow.shap.log_explanation function can be used with any model that has a predict function, such as scikit-learn, xgboost, etc1234 References:
• mlflow.shap — MLflow 2.9.1 documentation
• Extract feature importance from a mlflow 1.9 PyFuncModel model
• 9.6 SHAP (SHapley Additive exPlanations) | Interpretable Machine Learning
• load MLFlow model and plot feature importance with feature names