# Case Study: Relative Entropy Minimization

** Case study background and problem formulations**

**PROBLEM: problem_entropyr**

Minimize Entropyr (minimizing Relative Entropy)

subject to

Linear = 0 (constraint on sum of decision variables (probabilities))

A * x = b (linear equality constraint)

Box constraints (non-negative lower bound for probabilities)

——————————————————————–

Entropyr = Relative Entropy

A = matrix in the linear equality constraint

b = vector in the right hand side of the linear equality constraint

Box constraints = constraints on individual decision variables

——————————————————————–

Data and solution in Run-File Environment

Problem Datasets | # of Variables | # of Scenarios | Objective Value | Solving Time, PC 2.66GHz (sec) | |||
---|---|---|---|---|---|---|---|

Dataset1 | Problem Statement | Data | Solution | 250 | 3 | 2.167273 | <0.01 |

Dataset2 | Problem Statement | Data | Solution | 100,000 | 3 | 1.923068 | 1.38 |

Data and solution in MATLAB Environment

Problem Datasets | # of Variables | # of Scenarios | Objective Value | Solving Time, PC 2.66GHz (sec) | |||
---|---|---|---|---|---|---|---|

Dataset1 | Matlab code | Data | Solution | 250 | 3 | 2.167273 | <0.01 |

Dataset2 | Matlab code | Data | Solution | 100,000 | 3 | 1.92307 | 1.1 |

CASE STUDY SUMMARY

This case study demonstrates an optimization problem for minimizing Relative Entropy with linear constraints. Relative Entropy is used to find a probability distribution which is the most close to some “prior” probability distribution subject to available information about the distribution. For instance, moments of a distribution can be known and we want to find the “best” distribution accounting for this information.

Problem is solved for 250 probability atoms of a discrete probability distribution (250 decision variables), and for 100,000 probability atoms of a discrete probability distribution (100,000 decision variables).

Problem is solved using 2 datasets:

• Dataset1 for 250 decision variables;

• Dataset2 for 100,000 decision variables.