dmgp.models
Models for Fully-connected DMGP
GPLayer
- class dmgp.models.GPLayer(in_features, out_features, num_inducing=2, input_lb=-2, input_ub=2, dense=<class 'dmgp.layers.linear.LinearFlipout'>, gp_activation=<class 'dmgp.layers.activation.AMK'>, kernel=LaplaceProductKernel(), design_class=<class 'dmgp.utils.sparse_design.design_class.HyperbolicCrossDesign'>)[source]
Represents a layer in DMGP where inference is performed via finite-rank approximation, which can be represented as a one-layer neural network with correlated Gaussian distributed weights:
\[\begin{split}\begin{align*} \hat{\mathcal{G}}^{(i)}(\cdot) := & \mu + k(\cdot, \mathbf{U}) [ k(\mathbf{U}, \mathbf{U})]^{-1} \mathcal{G}^{(i)}(\mathbf{U}), \\ = & \mu + k(\cdot, \mathbf{U}) R^{-1}_{\mathbf{U}} \mathbf{Z} \\ = & \mu + \phi^{T}(\cdot) \mathbf{Z} \end{align*}\end{split}\]A GP Layer consists of a GP activation \(\phi(\cdot) = k(\cdot, \mathbf{U}) R^{-1}_{\mathbf{U}}\) and a linear layer with Gaussian weights \(\mathbf{Z} = [R^{T}_{\mathbf{U}}]^-1 \mathcal{G}(\mathbf{U})\). \(\mathbf{U}=\{ \mathbf{u}_i \}_{i=1}^{m}\) are the inducing points for approximating GP. \(R_{\mathbf{U}}\) is the Cholesky decomposition of the kernel matrix \(k(\mathbf{U}, \mathbf{U})\). The algorithm of Cholesky decomposition in DMGP can be found in dmgp.utils.operators.chol_inv.
- Parameters:
in_features (int) – Input features of \(\mathbf x_1\).
out_features (int) – Output features of GP layer.
num_inducing (int, optional) – Level of inducing points for approximating GP. Default: 3.
input_lb (float, optional) – Lower bound of the input space. You can choose any bound you want and apply normalization in the front. Default: -2..
input_ub (float, optional) – Upper bound of the input space. You can choose any bound you want and apply normalization in the front. Default: -2..
dense (class, dmgp.layers.linear, optional) – The dense linear layer for Gaussian weights. Default: LinearFlipout.
gp_activation (class, dmgp.layers.activation, optional) – The GP activation layer. Default: AMK.
kernel (class, dmgp.kernels, optional) – The GP kernel. Default: LaplaceProductKernel.
design_class (class, dmgp.utils.sparse_design.design_class, optional) – The design class of choosing inducing points for approximating GP. Default: HyperbolicCrossDesign.
DMGP
- class dmgp.models.DMGP(in_features, out_features, num_layers=2, hidden_dim=8, num_inducing=2, input_lb=-2, input_ub=2, kernel=LaplaceProductKernel(), design_class=<class 'dmgp.utils.sparse_design.design_class.HyperbolicCrossDesign'>, linear_layer=<class 'dmgp.layers.linear.LinearFlipout'>, option='additive')[source]
A container module to build a Deep GP. This module should contain GPLayer modules, and can also contain other modules as well.
- Parameters:
in_features (int) – Input features of \(\mathbf x_1\).
out_features (int) – Output features of GP layer.
num_layers (int) – Number of hidden layers in DMGP model. Default: 2.
hidden_dim (int) – Dimension of hidden layers in DMGP model. Default: 8.
num_inducing (int, optional) – Level of inducing points for approximating GP. For “sparse grid” design, we recommend low level of inducing. Default: 3.
input_lb (float, optional) – Lower bound of the input space. You can choose any bound you want and apply normalization in the front. Default: -2..
input_ub (float, optional) – Upper bound of the input space. You can choose any bound you want and apply normalization in the front. Default: -2..
kernel (class, dmgp.kernels, optional) – The GP kernel. Default: LaplaceProductKernel.
design_class (class, dmgp.utils.sparse_design.design_class, optional) – The design class of choosing inducing points for approximating GP. Default: HyperbolicCrossDesign.
linear_layer (class, dmgp.layers.linear, optional) – The dense linear layer for Gaussian weights. Default: LinearFlipout.
option (str, optional) – The option of DMGP architecture: use sparse grid or additive structure. Default: additive.