Support Vector Regression(SVR)
The Support Vector Regression (SVR) is adopted from support Vector Machine (SVM) for the regression type data to predict the value. While dealing with real number data, the SVM changes its variant as regression.
π(π₯) = πππ(π₯) + π ---------------(1)
where
βπ€ is the weight vector π dimension
βπ(π₯) is a function that maps π₯ to the feature space with π dimensions
β π is biased
βπ as the distance between the hyperplane and the two boundary lines
The coefficients π€ and π are estimated by minimizing the risk function. Therefore, to maximize the margin πΏ, a minimum βπ€β is needed. Optimization of problem-solving as shown in function (3.2).
πin1/2 βπ€βΒ² --------------------------(2)
with the condition, π¦π β π€ππ(π₯π) β π β€ π, πor π = 1, β¦ , π and π€ππ(π₯π) β π¦π + π β€ π, πor π = 1, β¦ , π. Where π¦π is the actual value of π period, and π(π₯π) is the estimated value of π period.
πin1/2 βπ€βΒ² + β ββπ=1 (ππ + ππβ) --------------------------(3.3)
with the condition
π¦π β π€ππ(π₯π) β π β ππ β€ π, πor π = 1, β¦ , π
π€ππ(π₯π) β π¦π + π- ππβ β€ π, πor π = 1, β¦ , π
ππ , ππβ β₯ 0
The βπ€βΒ² factor is called regulation. Minimizing βπ€βΒ² will make a function as thin (flat) as possible so that it can control the functional capacity. All points outside the margin/limit π will be penalized.
πΆ > 0 constant determines how much the error deviation is from the tolerable limit ππ. The formula above is a Convex Linear Programming NLP Optimization Problem which functions to minimize the quadratic function to be converted into a constraint. This limitation can be solved by using the Lagrange Multiplier function. The process of deriving formulas is very long and complicated. After going through mathematical stages, a new equation is obtained with the function:
π(π₯) = βli=1 (ππ β ππ β). (π₯π. π₯) + π ----(4)
Where xi is the support vector and π₯ is the test vector. The above functions can be used to solve linear problems. Whereas for non-linear problems the values of π₯π, and π₯ are first transformed into a high-dimensional feature space by mapping the vectors π₯π and π₯ into the kernel function so that the final function becomes:
π(π₯) = βli=1 (ππ β ππ β). k(π₯π. π₯) + π ----(4)
The function k(π₯π. π₯) is the Kernel. The table 1 below shows the kernels used in the SVR calculation (Scholkopf & Smola, 2018)