Back to main page
Case study background and problem formulations
Instructions for optimization with PSG Run-File, PSG MATLAB Toolbox, PSG MATLAB Subroutines and PSG R.
——————————————————————–———————————————————————————
At the link below, a Python script is available for calculating bAUC and plotting the bROC curve as described in “Maximization of AUC and Buffered AUC in Binary Classification”, by Matthew Norton & Stan Uryasev.
The Python script has the following dependencies (any recent versions of these packages will work):
Numpy, Gurobi, Gurobipy, Matplotlib, Scikit-Learn
——————————————————————–———————————————————————————
At the link below, a MATLAB script is available which demonstrates how to optimize bAUC as well as calculate bAUC using the same artificial data set used in Section 4.5.2 of “Maximization of AUC and Buffered AUC in Binary Classification.”
——————————————————————–———————————————————————————
Additional instructions for optimization with PSG Run-File, PSG MATLAB Toolbox and PSG MATLAB Subroutines.
——————————————————————–———————————————————————————
PROBLEM1: problem_bAUC_1
Minimizing Pm_pen (Buffered Probability of Exceedance)
——————————————————————–——————
Pm_pen = Partial Moment Penalty for Loss
——————————————————————–——————
Minimizing Pm_pen (Buffered Probability of Exceedance)
——————————————————————–——————
Pm_pen = Partial Moment Penalty for Loss
——————————————————————–——————
| # of Variables | # of Scenarios | Objective Value | Solving Time, PC 3.14GHz (sec) | ||||
| Dataset | 6 | 3990*2788=11,124,120 | 0.7354 | 0.13 | |||
|---|---|---|---|---|---|---|---|
| Environments | |||||||
| Run-File | Problem Statement | Data | Solution | ||||
| Matlab Toolbox | Data | ||||||
| Matlab Subroutines | Matlab Code | Data | |||||
| R | R Code | Data | |||||
PROBLEM2: problem_bAUC_2
Minimizing C1*Pm_pen_g + C2*Pm_pen (cost function)
——————————————————————–——————
Pm_pen = Partial Moment Penalty for Loss
Pm_pen_g = Partial Moment Penalty for Gain
——————————————————————–——————
Minimizing C1*Pm_pen_g + C2*Pm_pen (cost function)
——————————————————————–——————
Pm_pen = Partial Moment Penalty for Loss
Pm_pen_g = Partial Moment Penalty for Gain
——————————————————————–——————
| # of Variables | # of Scenarios | Objective Value | Solving Time, PC 3.14GHz (sec) | ||||
| Dataset | 1 | 3990*2788=11,124,120 | 1.7397 | <1 | |||
|---|---|---|---|---|---|---|---|
| Environments | |||||||
| Run-File | Problem Statement | Data | Solution | ||||
| Matlab Toolbox | Data | ||||||
| Matlab Subroutines | Matlab Code | Data | |||||
| R | R Code | Data | |||||
PROBLEM3: problem_bAUC_3
Solve problem for 7 values of alfaMinimizing C1*Pm_pen_g + C2*Pm_pen (cost function)
subject to
Pm_pen <= 1-alpha (constraint on bAUC)
——————————————————————–——————
Pm_pen = Partial Moment Penalty for Loss
Pm_pen_g = Partial Moment Penalty for Gain
——————————————————————–——————
Solve problem for 7 values of alfaMinimizing C1*Pm_pen_g + C2*Pm_pen (cost function)
subject to
Pm_pen <= 1-alpha (constraint on bAUC)
——————————————————————–——————
Pm_pen = Partial Moment Penalty for Loss
Pm_pen_g = Partial Moment Penalty for Gain
——————————————————————–——————
| # of Variables | # of Scenarios | Objective Value | Solving Time, PC 3.14GHz (sec) | ||||
| Dataset | 7 | 3990*2788=11,124,120 | 0.7354 | 0.22 | |||
|---|---|---|---|---|---|---|---|
| Environments | |||||||
| Run-File | Problem Statement | Data | Solution | ||||
| Matlab Toolbox | Data | ||||||
| Matlab Subroutines | Matlab Code | Data | |||||
| R | R Code | Data | |||||
CASE STUDY SUMMARY
This case study considers buffered version (bAUC) of classification criteria Area Under the Receiver Operating Characteristic Curve (AUC). Two optimization settings: 1) maximizing bAUC and then finding intercept by minimizing some cost function; 2) Construction of “Efficient Frontier” by minimizing cost function with several constraints on bAUC.
This case study considers buffered version (bAUC) of classification criteria Area Under the Receiver Operating Characteristic Curve (AUC). Two optimization settings: 1) maximizing bAUC and then finding intercept by minimizing some cost function; 2) Construction of “Efficient Frontier” by minimizing cost function with several constraints on bAUC.
References
• Norton, M. and S. Uryasev. Maximization of AUC and Buffered AUC in Classification. Research Report 2014-2, ISE Dept., University of Florida, October 2014.
• Norton, M. and S. Uryasev. Maximization of AUC and Buffered AUC in Classification. Research Report 2014-2, ISE Dept., University of Florida, October 2014.