Introduction
Optimization is a mathematical approach to deduce the best results from the constraints provided. You should identify the values of decision variable that maximizes or minimizes an objective function. This method has a wide applicability range, such as in engineering, economics, finance and also operations research.
Key Concepts in Optimization
Decision Variables: The parameters that can be changed to identify the best outcome.
Objective Function: The function we want to maximise or minimise
Constraints: The restrictions for decision variables
Feasible Region — The area of all possible solutions that fit the constraints.
Best Solution: A feasible solution that is the best with respect to both an objective function and constraints.
However, when we dive deeper into optimization problems we will find many different types of them
Linear Programming (LP):
Assumes linear objective function and linear constraints.
It’s especially useful for step-type problems such as resource allocation, production planning and transportation.
Integer Programming (IP):
LP on a special case where some or all of its decision variables must be integers
Applied to problems such as scheduling, facility location and network design.
Nonlinear Programming (NLP):
Has nonlinear objective functions and/or nonlinear constraints
Suitable for problems such as engineering design, chemical process optimization and portfolio optimization.
Dynamic Programming (DP):
A method for solving optimization problems by decomposing them into simpler subproblems.
Application: Used for problems such as optimal control, inventory management, and resource allocation.
Optimization Techniques
Analytical Methods:
Take derivative of the objective function and equate it to zero.
Numerical Methods:
Methods based on an iterative approximation regarding the optimal solution.
This can be done using lots of different techniques such as gradient descent, newton’s method or even simulated annealing.
Metaheuristic Algorithms:
Heuristic algorithms, which search the solution space for near-optimal solutions.
Common types of meta-heuristic methods are genetic algorithms, particle swarm optimization, and ant colony optimization.
Applications of Optimization
Engineering Design: Designing structures, machines and systems optimally
