skorch.probabilistic

Integrate GPyTorch for Gaussian Processes

• The criterion always takes likelihood and module as input arguments

• Always optimize the negative objective function

• Need elaboration on how batching works - are distributions disjoint?

class skorch.probabilistic.ExactGPRegressor(module, *args, likelihood=<class 'gpytorch.likelihoods.gaussian_likelihood.GaussianLikelihood'>, criterion=<class 'gpytorch.mlls.exact_marginal_log_likelihood.ExactMarginalLogLikelihood'>, batch_size=-1, **kwargs)

Exact Gaussian Process regressor

Use this specifically if you want to perform an exact solution to the Gaussian Process. This implies that the module should by a ExactGP module and you cannot use batching (i.e. batch size should be -1).

Parameters

Modulegpytorch.models.ExactGP (class or instance)

The module needs to return a MultivariateNormal distribution.

likelihoodgpytorch.likelihoods.GaussianLikelihood (class or instance)

The likelihood used for the exact GP regressor. Usually doesn’t need to be changed.

criteriongpytorch.mlls.ExactMarginalLogLikelihood

The objective function to learn the posterior of of the GP regressor. Usually doesn’t need to be changed.

optimizertorch optim (class, default=torch.optim.SGD)

The uninitialized optimizer (update rule) used to optimize the module

lrfloat (default=0.01)

Learning rate passed to the optimizer. You may use lr instead of using optimizer__lr, which would result in the same outcome.

max_epochsint (default=10)

The number of epochs to train for each fit call. Note that you may keyboard-interrupt training at any time.

batch_sizeint (default=-1)

Mini-batch size. For exact GPs, it must be set to -1, since the exact solution cannot deal with batching. To make use of batching, use GPRegressor in conjunction with a variational strategy.

iterator_traintorch DataLoader

The default PyTorch DataLoader used for training data.

iterator_validtorch DataLoader

The default PyTorch DataLoader used for validation and test data, i.e. during inference.

datasettorch Dataset (default=skorch.dataset.Dataset)

The dataset is necessary for the incoming data to work with pytorch’s DataLoader. It has to implement the __len__ and __getitem__ methods. The provided dataset should be capable of dealing with a lot of data types out of the box, so only change this if your data is not supported. You should generally pass the uninitialized Dataset class and define additional arguments to X and y by prefixing them with dataset__. It is also possible to pass an initialzed Dataset, in which case no additional arguments may be passed.

train_splitNone or callable (default=None)

If None, there is no train/validation split. Else, train_split should be a function or callable that is called with X and y data and should return the tuple dataset_train, dataset_valid. The validation data may be None. There is no default train split for GP regressors because random splitting is typically not desired, e.g. because there is a temporal relationship between samples.

callbacksNone, “disable”, or list of Callback instances (default=None)

Which callbacks to enable. There are three possible values:

If callbacks=None, only use default callbacks, those returned by get_default_callbacks.

If callbacks="disable", disable all callbacks, i.e. do not run any of the callbacks, not even the default callbacks.

If callbacks is a list of callbacks, use those callbacks in addition to the default callbacks. Each callback should be an instance of Callback.

Callback names are inferred from the class name. Name conflicts are resolved by appending a count suffix starting with 1, e.g. EpochScoring_1. Alternatively, a tuple (name, callback) can be passed, where name should be unique. Callbacks may or may not be instantiated. The callback name can be used to set parameters on specific callbacks (e.g., for the callback with name 'print_log', use net.set_params(callbacks__print_log__keys_ignored=['epoch', 'train_loss'])).

predict_nonlinearitycallable, None, or ‘auto’ (default=’auto’)

The nonlinearity to be applied to the prediction. When set to ‘auto’, infers the correct nonlinearity based on the criterion (softmax for CrossEntropyLoss and sigmoid for BCEWithLogitsLoss). If it cannot be inferred or if the parameter is None, just use the identity function. Don’t pass a lambda function if you want the net to be pickleable.

In case a callable is passed, it should accept the output of the module (the first output if there is more than one), which is a PyTorch tensor, and return the transformed PyTorch tensor.

This can be useful, e.g., when predict_proba() should return probabilities but a criterion is used that does not expect probabilities. In that case, the module can return whatever is required by the criterion and the predict_nonlinearity transforms this output into probabilities.

The nonlinearity is applied only when calling predict() or predict_proba() but not anywhere else – notably, the loss is unaffected by this nonlinearity.

warm_startbool (default=False)

Whether each fit call should lead to a re-initialization of the module (cold start) or whether the module should be trained further (warm start).

verboseint (default=1)

This parameter controls how much print output is generated by the net and its callbacks. By setting this value to 0, e.g. the summary scores at the end of each epoch are no longer printed. This can be useful when running a hyperparameter search. The summary scores are always logged in the history attribute, regardless of the verbose setting.

devicestr, torch.device, or None (default=’cpu’)

The compute device to be used. If set to ‘cuda’ in order to use GPU acceleration, data in torch tensors will be pushed to cuda tensors before being sent to the module. If set to None, then all compute devices will be left unmodified.

compilebool (default=False)

If set to True, compile all modules using torch.compile. For this to work, the installed torch version has to support torch.compile. Compiled modules should work identically to non-compiled modules but should run faster on new GPU architectures (Volta and Ampere for instance). Additional arguments for torch.compile can be passed using the dunder notation, e.g. when initializing the net with compile__dynamic=True, torch.compile will be called with dynamic=True.

use_cachingbool or ‘auto’ (default=’auto’)

Optionally override the caching behavior of scoring callbacks. Callbacks such as EpochScoring and BatchScoring allow to cache the inference call to save time when calculating scores during training at the expense of memory. In certain situations, e.g. when memory is tight, you may want to disable caching. As it is cumbersome to change the setting on each callback individually, this parameter allows to override their behavior globally. By default ('auto'), the callbacks will determine if caching is used or not. If this argument is set to False, caching will be disabled on all callbacks. If set to True, caching will be enabled on all callbacks. Implementation note: It is the job of the callbacks to honor this setting.

torch_load_kwargsdict or None (default=None)

Additional arguments that will be passed to torch.load when load pickled parameters.

In particular, this is important to because PyTorch will switch (probably in version 2.6.0) to only allow weights to be loaded for security reasons (i.e weights_only switches from False to True). As a consequence, loading pickled parameters may raise an error after upgrading torch because some types are used that are considered insecure. In skorch, we will also make that switch at the same time. To resolve the error, follow the instructions in the torch error message to designate the offending types as secure. Only do this if you trust the source of the file.

If you want to keep loading non-weight types the same way as before, please pass:

torch_load_kwargs={‘weights_only’: False}

You should be aware that this is considered insecure and should only be used if you trust the source of the file. However, this does not introduce new insecurities, it rather corresponds to the status quo from before torch made the switch.

Another way to avoid this issue is to pass use_safetensors=True when calling save_params and load_params. This avoid using pickle in favor of the safetensors format, which is secure by design.

Attributes

prefixes_list of str

Contains the prefixes to special parameters. E.g., since there is the 'optimizer' prefix, it is possible to set parameters like so: NeuralNet(..., optimizer__momentum=0.95). Some prefixes are populated dynamically, based on what modules and criteria are defined.

cuda_dependent_attributes_list of str

Contains a list of all attribute prefixes whose values depend on a CUDA device. If a NeuralNet trained with a CUDA-enabled device is unpickled on a machine without CUDA or with CUDA disabled, the listed attributes are mapped to CPU. Expand this list if you want to add other cuda-dependent attributes.

initialized_bool

Whether the NeuralNet was initialized.

module_torch module (instance)

The instantiated module.

criterion_torch criterion (instance)

The instantiated criterion.

callbacks_list of tuples

The complete (i.e. default and other), initialized callbacks, in a tuple with unique names.

_moduleslist of str

List of names of all modules that are torch modules. This list is collected dynamically when the net is initialized. Typically, there is no reason for a user to modify this list.

_criterialist of str

List of names of all criteria that are torch modules. This list is collected dynamically when the net is initialized. Typically, there is no reason for a user to modify this list.

_optimizerslist of str

List of names of all optimizers. This list is collected dynamically when the net is initialized. Typically, there is no reason for a user to modify this list.

likelihood_: torch module (instance)

The instantiated likelihood.

Methods

check_is_fitted([attributes])	Checks whether the GP is initialized.
check_training_readiness()	Check that the net is ready to train
confidence_region(X[, sigmas])	Returns 2 standard deviations above and below the mean.
evaluation_step(batch[, training])	Perform a forward step to produce the output used for prediction and scoring.
fit(X[, y])	Initialize and fit the module.
fit_loop(X[, y, epochs])	The proper fit loop.
forward(X[, training, device])	Gather and concatenate the output from forward call with input data.
forward_iter(X, *args, **kwargs)	Yield outputs of module forward calls on each batch of data.
get_all_learnable_params()	Yield the learnable parameters of all modules
get_dataset(X[, y])	Get a dataset that contains the input data and is passed to the iterator.
get_iterator(dataset[, training])	Get an iterator that allows to loop over the batches of the given data.
get_loss(y_pred, y_true[, X, training])	Return the loss for this batch.
get_params_for(prefix)	Collect and return init parameters for an attribute.
get_params_for_optimizer(prefix, ...)	Collect and return init parameters for an optimizer.
get_split_datasets(X[, y])	Get internal train and validation datasets.
get_train_step_accumulator()	Return the train step accumulator.
infer(x, **fit_params)	Perform a single inference step on a batch of data.
initialize()	Initializes all of its components and returns self.
initialize_callbacks()	Initializes all callbacks and save the result in the callbacks_ attribute.
initialize_criterion()	Initializes the criterion.
initialize_history()	Initializes the history.
initialize_module()	Initializes likelihood and module.
initialize_optimizer([triggered_directly])	Initialize the model optimizer.
initialized_instance(instance_or_cls, kwargs)	Return an instance initialized with the given parameters
load_params([f_params, f_optimizer, ...])	Loads the the module's parameters, history, and optimizer, not the whole object.
notify(method_name, **cb_kwargs)	Call the callback method specified in method_name with parameters specified in cb_kwargs.
on_batch_begin(net[, batch, training])
on_epoch_begin(net[, dataset_train, ...])
on_epoch_end(net[, dataset_train, dataset_valid])
on_train_begin(net[, X, y])
on_train_end(net[, X, y])
partial_fit(X[, y, classes])	Fit the module.
predict(X[, return_std, return_cov])	Returns the predicted mean and optionally standard deviation.
predict_proba(X)	Return the output of the module's forward method as a numpy array.
run_single_epoch(iterator, training, prefix, ...)	Compute a single epoch of train or validation.
sample(X, n_samples[, axis])	Return samples conditioned on input data.
save_params([f_params, f_optimizer, ...])	Saves the module's parameters, history, and optimizer, not the whole object.
set_params(**kwargs)	Set the parameters of this class.
torch_compile(module, name)	Compile torch modules
train_step(batch, **fit_params)	Prepares a loss function callable and pass it to the optimizer, hence performing one optimization step.
train_step_single(batch, **fit_params)	Compute y_pred, loss value, and update net's gradients.
trim_for_prediction()	Remove all attributes not required for prediction.
validation_step(batch, **fit_params)	Perform a forward step using batched data and return the resulting loss.

check_data
get_default_callbacks
get_params
initialize_virtual_params
on_batch_end
on_grad_computed

fit(X, y=None, **fit_params)

Initialize and fit the module.

If the module was already initialized, by calling fit, the module will be re-initialized (unless warm_start is True).

Parameters

Xinput data, compatible with skorch.dataset.Dataset

By default, you should be able to pass:

• numpy arrays

• torch tensors

• pandas DataFrame or Series

• scipy sparse CSR matrices

• a dictionary of the former three

• a list/tuple of the former three

• a Dataset

If this doesn’t work with your data, you have to pass a Dataset that can deal with the data.

ytarget data, compatible with skorch.dataset.Dataset

The same data types as for X are supported. If your X is a Dataset that contains the target, y may be set to None.

**fit_paramsdict

Additional parameters passed to the forward method of the module and to the self.train_split call.

initialize_module()

Initializes likelihood and module.

class skorch.probabilistic.GPBinaryClassifier(module, *args, likelihood=<class 'gpytorch.likelihoods.bernoulli_likelihood.BernoulliLikelihood'>, criterion=<class 'gpytorch.mlls.variational_elbo.VariationalELBO'>, train_split=<skorch.dataset.ValidSplit object>, threshold=0.5, **kwargs)

Gaussian Process binary classifier

Use this for variational and approximate Gaussian process binary classification. This implies that the module should by a ApproximateGP module.

Parameters

Modulegpytorch.models.ApproximateGP (class or instance)

The GPyTorch module; in contrast to exact GP, the return distribution does not need to be Gaussian.

likelihoodgpytorch.likelihoods.BernoulliLikelihood (class or instance)

The likelihood used for the exact GP binary classification. Usually doesn’t need to be changed.

criteriongpytorch.mlls.VariationalELBO

The objective function to learn the approximate posterior of of the GP binary classification.

optimizertorch optim (class, default=torch.optim.SGD)

The uninitialized optimizer (update rule) used to optimize the module

lrfloat (default=0.01)

Learning rate passed to the optimizer. You may use lr instead of using optimizer__lr, which would result in the same outcome.

max_epochsint (default=10)

The number of epochs to train for each fit call. Note that you may keyboard-interrupt training at any time.

batch_sizeint (default=128)

Mini-batch size. Use this instead of setting iterator_train__batch_size and iterator_test__batch_size, which would result in the same outcome. If batch_size is -1, a single batch with all the data will be used during training and validation.

iterator_traintorch DataLoader

The default PyTorch DataLoader used for training data.

iterator_validtorch DataLoader

The default PyTorch DataLoader used for validation and test data, i.e. during inference.

datasettorch Dataset (default=skorch.dataset.Dataset)

The dataset is necessary for the incoming data to work with pytorch’s DataLoader. It has to implement the __len__ and __getitem__ methods. The provided dataset should be capable of dealing with a lot of data types out of the box, so only change this if your data is not supported. You should generally pass the uninitialized Dataset class and define additional arguments to X and y by prefixing them with dataset__. It is also possible to pass an initialzed Dataset, in which case no additional arguments may be passed.

train_splitNone or callable (default=skorch.dataset.ValidSplit(5))

If None, there is no train/validation split. Else, train_split should be a function or callable that is called with X and y data and should return the tuple dataset_train, dataset_valid. The validation data may be None.

callbacksNone, “disable”, or list of Callback instances (default=None)

Which callbacks to enable. There are three possible values:

If callbacks=None, only use default callbacks, those returned by get_default_callbacks.

If callbacks="disable", disable all callbacks, i.e. do not run any of the callbacks, not even the default callbacks.

If callbacks is a list of callbacks, use those callbacks in addition to the default callbacks. Each callback should be an instance of Callback.

Callback names are inferred from the class name. Name conflicts are resolved by appending a count suffix starting with 1, e.g. EpochScoring_1. Alternatively, a tuple (name, callback) can be passed, where name should be unique. Callbacks may or may not be instantiated. The callback name can be used to set parameters on specific callbacks (e.g., for the callback with name 'print_log', use net.set_params(callbacks__print_log__keys_ignored=['epoch', 'train_loss'])).

predict_nonlinearitycallable, None, or ‘auto’ (default=’auto’)

The nonlinearity to be applied to the prediction. When set to ‘auto’, infers the correct nonlinearity based on the criterion (softmax for CrossEntropyLoss and sigmoid for BCEWithLogitsLoss). If it cannot be inferred or if the parameter is None, just use the identity function. Don’t pass a lambda function if you want the net to be pickleable.

In case a callable is passed, it should accept the output of the module (the first output if there is more than one), which is a PyTorch tensor, and return the transformed PyTorch tensor.

This can be useful, e.g., when predict_proba() should return probabilities but a criterion is used that does not expect probabilities. In that case, the module can return whatever is required by the criterion and the predict_nonlinearity transforms this output into probabilities.

The nonlinearity is applied only when calling predict() or predict_proba() but not anywhere else – notably, the loss is unaffected by this nonlinearity.

warm_startbool (default=False)

Whether each fit call should lead to a re-initialization of the module (cold start) or whether the module should be trained further (warm start).

verboseint (default=1)

This parameter controls how much print output is generated by the net and its callbacks. By setting this value to 0, e.g. the summary scores at the end of each epoch are no longer printed. This can be useful when running a hyperparameter search. The summary scores are always logged in the history attribute, regardless of the verbose setting.

devicestr, torch.device, or None (default=’cpu’)

The compute device to be used. If set to ‘cuda’ in order to use GPU acceleration, data in torch tensors will be pushed to cuda tensors before being sent to the module. If set to None, then all compute devices will be left unmodified.

compilebool (default=False)

If set to True, compile all modules using torch.compile. For this to work, the installed torch version has to support torch.compile. Compiled modules should work identically to non-compiled modules but should run faster on new GPU architectures (Volta and Ampere for instance). Additional arguments for torch.compile can be passed using the dunder notation, e.g. when initializing the net with compile__dynamic=True, torch.compile will be called with dynamic=True.

use_cachingbool or ‘auto’ (default=’auto’)

Optionally override the caching behavior of scoring callbacks. Callbacks such as EpochScoring and BatchScoring allow to cache the inference call to save time when calculating scores during training at the expense of memory. In certain situations, e.g. when memory is tight, you may want to disable caching. As it is cumbersome to change the setting on each callback individually, this parameter allows to override their behavior globally. By default ('auto'), the callbacks will determine if caching is used or not. If this argument is set to False, caching will be disabled on all callbacks. If set to True, caching will be enabled on all callbacks. Implementation note: It is the job of the callbacks to honor this setting.

torch_load_kwargsdict or None (default=None)

Additional arguments that will be passed to torch.load when load pickled parameters.

In particular, this is important to because PyTorch will switch (probably in version 2.6.0) to only allow weights to be loaded for security reasons (i.e weights_only switches from False to True). As a consequence, loading pickled parameters may raise an error after upgrading torch because some types are used that are considered insecure. In skorch, we will also make that switch at the same time. To resolve the error, follow the instructions in the torch error message to designate the offending types as secure. Only do this if you trust the source of the file.

If you want to keep loading non-weight types the same way as before, please pass:

torch_load_kwargs={‘weights_only’: False}

You should be aware that this is considered insecure and should only be used if you trust the source of the file. However, this does not introduce new insecurities, it rather corresponds to the status quo from before torch made the switch.

Another way to avoid this issue is to pass use_safetensors=True when calling save_params and load_params. This avoid using pickle in favor of the safetensors format, which is secure by design.

Attributes

prefixes_list of str

Contains the prefixes to special parameters. E.g., since there is the 'optimizer' prefix, it is possible to set parameters like so: NeuralNet(..., optimizer__momentum=0.95). Some prefixes are populated dynamically, based on what modules and criteria are defined.

cuda_dependent_attributes_list of str

Contains a list of all attribute prefixes whose values depend on a CUDA device. If a NeuralNet trained with a CUDA-enabled device is unpickled on a machine without CUDA or with CUDA disabled, the listed attributes are mapped to CPU. Expand this list if you want to add other cuda-dependent attributes.

initialized_bool

Whether the NeuralNet was initialized.

module_torch module (instance)

The instantiated module.

criterion_torch criterion (instance)

The instantiated criterion.

callbacks_list of tuples

The complete (i.e. default and other), initialized callbacks, in a tuple with unique names.

_moduleslist of str

List of names of all modules that are torch modules. This list is collected dynamically when the net is initialized. Typically, there is no reason for a user to modify this list.

_criterialist of str

List of names of all criteria that are torch modules. This list is collected dynamically when the net is initialized. Typically, there is no reason for a user to modify this list.

_optimizerslist of str

List of names of all optimizers. This list is collected dynamically when the net is initialized. Typically, there is no reason for a user to modify this list.

likelihood_: torch module (instance)

The instantiated likelihood.

Methods

check_data(X, y)
check_is_fitted([attributes])	Checks whether the GP is initialized.
check_training_readiness()	Check that the net is ready to train
confidence_region(X[, sigmas])	Returns 2 standard deviations above and below the mean.
evaluation_step(batch[, training])	Perform a forward step to produce the output used for prediction and scoring.
fit(X[, y])	Initialize and fit the module.
fit_loop(X[, y, epochs])	The proper fit loop.
forward(X[, training, device])	Gather and concatenate the output from forward call with input data.
forward_iter(X, *args, **kwargs)	Yield outputs of module forward calls on each batch of data.
get_all_learnable_params()	Yield the learnable parameters of all modules
get_dataset(X[, y])	Get a dataset that contains the input data and is passed to the iterator.
get_iterator(dataset[, training])	Get an iterator that allows to loop over the batches of the given data.
get_loss(y_pred, y_true[, X, training])	Return the loss for this batch.
get_params_for(prefix)	Collect and return init parameters for an attribute.
get_params_for_optimizer(prefix, ...)	Collect and return init parameters for an optimizer.
get_split_datasets(X[, y])	Get internal train and validation datasets.
get_train_step_accumulator()	Return the train step accumulator.
infer(x, **fit_params)	Perform a single inference step on a batch of data.
initialize()	Initializes all of its components and returns self.
initialize_callbacks()	Initializes all callbacks and save the result in the callbacks_ attribute.
initialize_criterion()	Initializes the criterion.
initialize_history()	Initializes the history.
initialize_module()	Initializes likelihood and module.
initialize_optimizer([triggered_directly])	Initialize the model optimizer.
initialized_instance(instance_or_cls, kwargs)	Return an instance initialized with the given parameters
load_params([f_params, f_optimizer, ...])	Loads the the module's parameters, history, and optimizer, not the whole object.
notify(method_name, **cb_kwargs)	Call the callback method specified in method_name with parameters specified in cb_kwargs.
on_batch_begin(net[, batch, training])
on_epoch_begin(net[, dataset_train, ...])
on_epoch_end(net[, dataset_train, dataset_valid])
on_train_begin(net[, X, y])
on_train_end(net[, X, y])
partial_fit(X[, y, classes])	Fit the module.
predict(X)	Return class labels for samples in X.
predict_proba(X)	Return probability estimates for the samples.
run_single_epoch(iterator, training, prefix, ...)	Compute a single epoch of train or validation.
sample(X, n_samples[, axis])	Return samples conditioned on input data.
save_params([f_params, f_optimizer, ...])	Saves the module's parameters, history, and optimizer, not the whole object.
set_params(**kwargs)	Set the parameters of this class.
torch_compile(module, name)	Compile torch modules
train_step(batch, **fit_params)	Prepares a loss function callable and pass it to the optimizer, hence performing one optimization step.
train_step_single(batch, **fit_params)	Compute y_pred, loss value, and update net's gradients.
trim_for_prediction()	Remove all attributes not required for prediction.
validation_step(batch, **fit_params)	Perform a forward step using batched data and return the resulting loss.

get_default_callbacks
get_params
initialize_virtual_params
on_batch_end
on_grad_computed

predict(X)

Return class labels for samples in X.

If the module’s forward method returns multiple outputs as a tuple, it is assumed that the first output contains the relevant information and the other values are ignored. If all values are relevant, consider using forward() instead.

Parameters

Xinput data, compatible with skorch.dataset.Dataset

By default, you should be able to pass:

• numpy arrays

• torch tensors

• pandas DataFrame or Series

• scipy sparse CSR matrices

• a dictionary of the former three

• a list/tuple of the former three

• a Dataset

If this doesn’t work with your data, you have to pass a Dataset that can deal with the data.

Returns

y_prednumpy ndarray

Predicted target values for X.

predict_proba(X)

Return probability estimates for the samples.

If the module’s forward method returns multiple outputs as a tuple, it is assumed that the first output contains the relevant information and the other values are ignored. If all values are relevant, consider using forward() instead.

Parameters

Xinput data, compatible with skorch.dataset.Dataset

By default, you should be able to pass:

• numpy arrays

• torch tensors

• pandas DataFrame or Series

• scipy sparse CSR matrices

• a dictionary of the former three

• a list/tuple of the former three

• a Dataset

If this doesn’t work with your data, you have to pass a Dataset that can deal with the data.

Returns

y_probanumpy ndarray

Probabilities for the samples, with the first column corresponding to class 0 and the second to class 1.

class skorch.probabilistic.GPRegressor(module, *args, likelihood=<class 'gpytorch.likelihoods.gaussian_likelihood.GaussianLikelihood'>, criterion=<class 'gpytorch.mlls.variational_elbo.VariationalELBO'>, **kwargs)

Gaussian Process regressor

Use this for variational and approximate Gaussian process regression. This implies that the module should by a ApproximateGP module.

Parameters

Modulegpytorch.models.ApproximateGP (class or instance)

The GPyTorch module; in contrast to exact GP, the return distribution does not need to be Gaussian.

likelihoodgpytorch.likelihoods.GaussianLikelihood (class or instance)

The likelihood used for the exact GP regressor. Usually doesn’t need to be changed.

criteriongpytorch.mlls.VariationalELBO

The objective function to learn the approximate posterior of of the GP regressor.

optimizertorch optim (class, default=torch.optim.SGD)

The uninitialized optimizer (update rule) used to optimize the module

lrfloat (default=0.01)

Learning rate passed to the optimizer. You may use lr instead of using optimizer__lr, which would result in the same outcome.

max_epochsint (default=10)

The number of epochs to train for each fit call. Note that you may keyboard-interrupt training at any time.

batch_sizeint (default=128)

Mini-batch size. Use this instead of setting iterator_train__batch_size and iterator_test__batch_size, which would result in the same outcome. If batch_size is -1, a single batch with all the data will be used during training and validation.

iterator_traintorch DataLoader

The default PyTorch DataLoader used for training data.

iterator_validtorch DataLoader

The default PyTorch DataLoader used for validation and test data, i.e. during inference.

datasettorch Dataset (default=skorch.dataset.Dataset)

The dataset is necessary for the incoming data to work with pytorch’s DataLoader. It has to implement the __len__ and __getitem__ methods. The provided dataset should be capable of dealing with a lot of data types out of the box, so only change this if your data is not supported. You should generally pass the uninitialized Dataset class and define additional arguments to X and y by prefixing them with dataset__. It is also possible to pass an initialzed Dataset, in which case no additional arguments may be passed.

train_splitNone or callable (default=None)

If None, there is no train/validation split. Else, train_split should be a function or callable that is called with X and y data and should return the tuple dataset_train, dataset_valid. The validation data may be None. There is no default train split for GP regressors because random splitting is typically not desired, e.g. because there is a temporal relationship between samples.

callbacksNone, “disable”, or list of Callback instances (default=None)

Which callbacks to enable. There are three possible values:

If callbacks=None, only use default callbacks, those returned by get_default_callbacks.

If callbacks="disable", disable all callbacks, i.e. do not run any of the callbacks, not even the default callbacks.

If callbacks is a list of callbacks, use those callbacks in addition to the default callbacks. Each callback should be an instance of Callback.

Callback names are inferred from the class name. Name conflicts are resolved by appending a count suffix starting with 1, e.g. EpochScoring_1. Alternatively, a tuple (name, callback) can be passed, where name should be unique. Callbacks may or may not be instantiated. The callback name can be used to set parameters on specific callbacks (e.g., for the callback with name 'print_log', use net.set_params(callbacks__print_log__keys_ignored=['epoch', 'train_loss'])).

predict_nonlinearitycallable, None, or ‘auto’ (default=’auto’)

The nonlinearity to be applied to the prediction. When set to ‘auto’, infers the correct nonlinearity based on the criterion (softmax for CrossEntropyLoss and sigmoid for BCEWithLogitsLoss). If it cannot be inferred or if the parameter is None, just use the identity function. Don’t pass a lambda function if you want the net to be pickleable.

In case a callable is passed, it should accept the output of the module (the first output if there is more than one), which is a PyTorch tensor, and return the transformed PyTorch tensor.

This can be useful, e.g., when predict_proba() should return probabilities but a criterion is used that does not expect probabilities. In that case, the module can return whatever is required by the criterion and the predict_nonlinearity transforms this output into probabilities.

The nonlinearity is applied only when calling predict() or predict_proba() but not anywhere else – notably, the loss is unaffected by this nonlinearity.

warm_startbool (default=False)

Whether each fit call should lead to a re-initialization of the module (cold start) or whether the module should be trained further (warm start).

verboseint (default=1)

This parameter controls how much print output is generated by the net and its callbacks. By setting this value to 0, e.g. the summary scores at the end of each epoch are no longer printed. This can be useful when running a hyperparameter search. The summary scores are always logged in the history attribute, regardless of the verbose setting.

devicestr, torch.device, or None (default=’cpu’)

The compute device to be used. If set to ‘cuda’ in order to use GPU acceleration, data in torch tensors will be pushed to cuda tensors before being sent to the module. If set to None, then all compute devices will be left unmodified.

compilebool (default=False)

If set to True, compile all modules using torch.compile. For this to work, the installed torch version has to support torch.compile. Compiled modules should work identically to non-compiled modules but should run faster on new GPU architectures (Volta and Ampere for instance). Additional arguments for torch.compile can be passed using the dunder notation, e.g. when initializing the net with compile__dynamic=True, torch.compile will be called with dynamic=True.

use_cachingbool or ‘auto’ (default=’auto’)

Optionally override the caching behavior of scoring callbacks. Callbacks such as EpochScoring and BatchScoring allow to cache the inference call to save time when calculating scores during training at the expense of memory. In certain situations, e.g. when memory is tight, you may want to disable caching. As it is cumbersome to change the setting on each callback individually, this parameter allows to override their behavior globally. By default ('auto'), the callbacks will determine if caching is used or not. If this argument is set to False, caching will be disabled on all callbacks. If set to True, caching will be enabled on all callbacks. Implementation note: It is the job of the callbacks to honor this setting.

torch_load_kwargsdict or None (default=None)

Additional arguments that will be passed to torch.load when load pickled parameters.

In particular, this is important to because PyTorch will switch (probably in version 2.6.0) to only allow weights to be loaded for security reasons (i.e weights_only switches from False to True). As a consequence, loading pickled parameters may raise an error after upgrading torch because some types are used that are considered insecure. In skorch, we will also make that switch at the same time. To resolve the error, follow the instructions in the torch error message to designate the offending types as secure. Only do this if you trust the source of the file.

If you want to keep loading non-weight types the same way as before, please pass:

torch_load_kwargs={‘weights_only’: False}

You should be aware that this is considered insecure and should only be used if you trust the source of the file. However, this does not introduce new insecurities, it rather corresponds to the status quo from before torch made the switch.

Another way to avoid this issue is to pass use_safetensors=True when calling save_params and load_params. This avoid using pickle in favor of the safetensors format, which is secure by design.

Attributes

prefixes_list of str

Contains the prefixes to special parameters. E.g., since there is the 'optimizer' prefix, it is possible to set parameters like so: NeuralNet(..., optimizer__momentum=0.95). Some prefixes are populated dynamically, based on what modules and criteria are defined.

cuda_dependent_attributes_list of str

Contains a list of all attribute prefixes whose values depend on a CUDA device. If a NeuralNet trained with a CUDA-enabled device is unpickled on a machine without CUDA or with CUDA disabled, the listed attributes are mapped to CPU. Expand this list if you want to add other cuda-dependent attributes.

initialized_bool

Whether the NeuralNet was initialized.

module_torch module (instance)

The instantiated module.

criterion_torch criterion (instance)

The instantiated criterion.

callbacks_list of tuples

The complete (i.e. default and other), initialized callbacks, in a tuple with unique names.

_moduleslist of str

List of names of all modules that are torch modules. This list is collected dynamically when the net is initialized. Typically, there is no reason for a user to modify this list.

_criterialist of str

List of names of all criteria that are torch modules. This list is collected dynamically when the net is initialized. Typically, there is no reason for a user to modify this list.

_optimizerslist of str

List of names of all optimizers. This list is collected dynamically when the net is initialized. Typically, there is no reason for a user to modify this list.

likelihood_: torch module (instance)

The instantiated likelihood.

Methods

check_is_fitted([attributes])	Checks whether the GP is initialized.
check_training_readiness()	Check that the net is ready to train
confidence_region(X[, sigmas])	Returns 2 standard deviations above and below the mean.
evaluation_step(batch[, training])	Perform a forward step to produce the output used for prediction and scoring.
fit(X[, y])	Initialize and fit the module.
fit_loop(X[, y, epochs])	The proper fit loop.
forward(X[, training, device])	Gather and concatenate the output from forward call with input data.
forward_iter(X, *args, **kwargs)	Yield outputs of module forward calls on each batch of data.
get_all_learnable_params()	Yield the learnable parameters of all modules
get_dataset(X[, y])	Get a dataset that contains the input data and is passed to the iterator.
get_iterator(dataset[, training])	Get an iterator that allows to loop over the batches of the given data.
get_loss(y_pred, y_true[, X, training])	Return the loss for this batch.
get_params_for(prefix)	Collect and return init parameters for an attribute.
get_params_for_optimizer(prefix, ...)	Collect and return init parameters for an optimizer.
get_split_datasets(X[, y])	Get internal train and validation datasets.
get_train_step_accumulator()	Return the train step accumulator.
infer(x, **fit_params)	Perform a single inference step on a batch of data.
initialize()	Initializes all of its components and returns self.
initialize_callbacks()	Initializes all callbacks and save the result in the callbacks_ attribute.
initialize_criterion()	Initializes the criterion.
initialize_history()	Initializes the history.
initialize_module()	Initializes likelihood and module.
initialize_optimizer([triggered_directly])	Initialize the model optimizer.
initialized_instance(instance_or_cls, kwargs)	Return an instance initialized with the given parameters
load_params([f_params, f_optimizer, ...])	Loads the the module's parameters, history, and optimizer, not the whole object.
notify(method_name, **cb_kwargs)	Call the callback method specified in method_name with parameters specified in cb_kwargs.
on_batch_begin(net[, batch, training])
on_epoch_begin(net[, dataset_train, ...])
on_epoch_end(net[, dataset_train, dataset_valid])
on_train_begin(net[, X, y])
on_train_end(net[, X, y])
partial_fit(X[, y, classes])	Fit the module.
predict(X[, return_std, return_cov])	Returns the predicted mean and optionally standard deviation.
predict_proba(X)	Return the output of the module's forward method as a numpy array.
run_single_epoch(iterator, training, prefix, ...)	Compute a single epoch of train or validation.
sample(X, n_samples[, axis])	Return samples conditioned on input data.
save_params([f_params, f_optimizer, ...])	Saves the module's parameters, history, and optimizer, not the whole object.
set_params(**kwargs)	Set the parameters of this class.
torch_compile(module, name)	Compile torch modules
train_step(batch, **fit_params)	Prepares a loss function callable and pass it to the optimizer, hence performing one optimization step.
train_step_single(batch, **fit_params)	Compute y_pred, loss value, and update net's gradients.
trim_for_prediction()	Remove all attributes not required for prediction.
validation_step(batch, **fit_params)	Perform a forward step using batched data and return the resulting loss.

check_data
get_default_callbacks
get_params
initialize_virtual_params
on_batch_end
on_grad_computed