multigrate.module.MultiVAETorch#
- class multigrate.module.MultiVAETorch(modality_lengths, condition_encoders=False, condition_decoders=True, normalization='layer', z_dim=16, losses=None, dropout=0.2, cond_dim=16, kernel_type='gaussian', loss_coefs=None, num_groups=1, integrate_on_idx=None, cat_covariate_dims=None, cont_covariate_dims=None, cat_covs_idx=None, cont_covs_idx=None, cont_cov_type='sigm', n_layers_cont_embed=1, n_layers_encoders=None, n_layers_decoders=None, n_hidden_cont_embed=16, n_hidden_encoders=None, n_hidden_decoders=None, modality_alignment=None, alignment_type='latent', activation='leaky_relu', initialization=None, mix='product')#
Bases:
BaseModuleClassMultigrate’s multimodal integration module.
- Parameters:
modality_lengths – List with lengths of each modality.
condition_encoders (default:
False) – Boolean to indicate if to condition encoders.condition_decoders (default:
True) – Boolean to indicate if to condition decoders.normalization (
Optional[Literal['layer','batch']] (default:'layer')) – One of the following *'layer'- layer normalization *'batch'- batch normalization *None- no normalization.z_dim (default:
16) – Dimensionality of the latent space.losses (default:
None) – List of which losses to use. For each modality can be one of the following: *'mse'- mean squared error *'nb'- negative binomial *'zinb'- zero-inflated negative binomial *'bce'- binary cross-entropy.dropout (default:
0.2) – Dropout rate for neural networks.cond_dim (default:
16) – Dimensionality of the covariate embeddings.kernel_type (
Optional[Literal['gaussian','not gaussian']] (default:'gaussian')) – One of the following: *'gaussian'- Gaussian kernel *'not gaussian'- not Gaussian kernel.loss_coefs (default:
None) – Dictionary with weights for each of the losses.num_groups (default:
1) – Number of groups to integrate on.integrate_on_idx (default:
None) – Indices on which to integrate on.cat_covariate_dims (default:
None) – List with number of classes for each of the categorical covariates.cont_covariate_dims (default:
None) – List of 1’s for each of the continuous covariate.cont_cov_type (
Optional[Literal['logsigm','sigm','mlp']] (default:'sigm')) – How to transform continuous covariate before multiplying with the embedding. One of the following: *'logsigm'- log generalized sigmoid, use for positive continuous covariates. *'sigm'- generalized sigmoid, use for continuous covariates that can take negative values. *'mlp'- MLP.n_layers_cont_embed (
int(default:1)) – Number of layers for the transformation of the continuous covariates before multiplying with the embedding.n_hidden_cont_embed (
int(default:16)) – Number of nodes in hidden layers in the network that transforms continuous covariates.n_layers_encoders (default:
None) – Number of layers in each encoder.n_layers_decoders (default:
None) – Number of layers in each decoder.n_hidden_encoders (default:
None) – Number of nodes in hidden layers in encoders.n_hidden_decoders (default:
None) – Number of nodes in hidden layers in decoders.modality_alignment (default:
None) – How to perform modality alignment. One of the following: *'MMD'- Maximum Mean Discrepancy *'Jeffreys'- Jeffreys Divergence *None- no modality alignment.alignment_type (default:
'latent') – How to calculate integration loss. One of the following: *'latent'- only on the latent representations *'marginal'- only on the marginal representations *'both'- the sum of the two above.activation (default:
'leaky_relu') – Activation function to use. One of the following: *'leaky_relu'- Leaky ReLU *'tanh'- Tanh.initialization (default:
None) – Weight initialization strategy. One of the following: *'xavier'- Xavier initialization *'kaiming'- Kaiming initialization *None- no initialization.mix (default:
'product') – How to combine modalities in the joint latent space. One of the following: *'product'- product of experts *'mixture'- mixture of experts.
Attributes table#
Methods table#
|
Compute necessary inference quantities. |
|
Compute necessary inference quantities. |
|
Calculate the (modality) reconstruction loss, Kullback divergences and integration loss. |
Select losses to plot. |
Attributes#
- MultiVAETorch.T_destination = ~T_destination#
- MultiVAETorch.call_super_init: bool = False#
- MultiVAETorch.device#
- MultiVAETorch.dump_patches: bool = False#
- MultiVAETorch.training: bool#
Methods#
- MultiVAETorch.generative(z_joint, cat_covs=None, cont_covs=None)#
Compute necessary inference quantities.
- Parameters:
- Return type:
- Returns:
Reconstructed values for each modality.
- MultiVAETorch.inference(x, cat_covs=None, cont_covs=None, masks=None)#
Compute necessary inference quantities.
- Parameters:
x (
Tensor) – Tensor of values with shape(batch_size, n_input_features).cat_covs (
Tensor|None(default:None)) – Categorical covariates to condition on.cont_covs (
Tensor|None(default:None)) – Continuous covariates to condition on.masks (
list[Tensor] |None(default:None)) – List of binary tensors indicating which values inxbelong to which modality.
- Return type:
- Returns:
Joint representations, marginal representations, joint mu’s and logvar’s.
- MultiVAETorch.loss(tensors, inference_outputs, generative_outputs, kl_weight=1.0)#
Calculate the (modality) reconstruction loss, Kullback divergences and integration loss.
- Parameters:
tensors – Tensor of values with shape
(batch_size, n_input_features).inference_outputs – Dictionary with the inference output.
generative_outputs – Dictionary with the generative output.
kl_weight (
float(default:1.0)) – Weight of the KL loss. Default is 1.0.
- Return type:
tuple[FloatTensor,dict[str,FloatTensor],FloatTensor,FloatTensor,FloatTensor,FloatTensor,dict[str,FloatTensor]]- Returns:
Reconstruction loss, Kullback divergences, integration loss and modality reconstruction losses.
- MultiVAETorch.select_losses_to_plot()#
Select losses to plot.
- Returns:
Loss names.