Basics

Introduction

PySCIPOpt-ML is a Python package for the automatic formulation of machine learning (ML) models into Mixed-Integer Programs (MIPs) using SCIP. This automatic formulation allows users to easily optimise mathematical optimisation problems with embedded ML constraints, without worrying about the exact formulation and how to extract the data from the ML interface.

The package currently supports various scikit-learn objects. It can also embed gradient boosting regression models from XGBoost, and LightGBM. Finally, it supports Sequential Neural Networks from PyTorch, Keras, and ONNX.

The package is actively developed and users are encouraged to raise an issue on GitHub if there are ML models that are currently available, or there are optimisation related problems with the created MIPs.

PySCIPOpt-ML flowchart

Install

We encourage to install the package via pip (or add it to your requirements.txt file):

(venv) pip install pyscipopt-ml

Note

If not already installed, this should install the pyscipopt and numpy packages.

Note

The package can also be installed from source. To do so first clone the package and then run:

python -m pip install .

Note

The following table lists the version of the relevant packages that are tested and supported.

Package

pyscipopt

numpy

torch

tensorflow

scikit-learn

lightgbm

xgboost

onnx

Installing any of the machine learning packages is only required if the predictor you want to insert uses them (i.e. to insert a Scikit-Learn based predictor you need to have scikit-learn installed).

Usage

The main function provided by the package is pyscipopt_ml.add_predictor_constr(). It takes as arguments: a PySCIPOpt Model, a supported ML model, input PySCIPOpt variables, and output PySCIPOpt variables.

By calling the function, the PySCIPOpt Model is augmented with variables and constraints so that, in a solution, the values of the output variables are predicted by the regression model from the values of the input variables. More formally, if we denote by \(g\) the prediction function of the embedded ML model, by \(x\) the input variables and by \(y\) the output variables, then \(y = g(x)\) in any solution.

The function add_predictor_constr returns a modeling object derived from the class AbstractPredictorConstr. That object keeps track of all the variables and constraints that have been added to the PySCIPOpt to establish the relationship between input and output variables of the ML model.

The modeling object can perform a few tasks:

  • It can print a summary of what it added with the print_stats method.

  • Once SCIP computed a solution to the optimization problem, it can compute the difference between what the ML model predicts from the input values and the values of the output variables in SCIP’s solution with the get_error method.

The function add_predictor_constr is a shorthand that should add the correct model for any supported ML model, but individual functions for each ML model are also available. For the list of frameworks and ML models supported, and the corresponding functions please refer to the supported section. We also briefly outline how the various ML models are formulated in SCIP in the Mixed Integer Formulations section.

For examples on how to use the package please refer to the the example.