NeuralNet subclasses for regression tasks.
NeuralNetRegressor(module, *args, criterion=<class 'torch.nn.modules.loss.MSELoss'>, **kwargs)¶
NeuralNet for regression tasks
Use this specifically if you have a standard regression task, with input data X and target y. y must be 2d.
In addition to the parameters listed below, there are parameters with specific prefixes that are handled separately. To illustrate this, here is an example:
>>> net = NeuralNet( ... ..., ... optimizer=torch.optimizer.SGD, ... optimizer__momentum=0.95, ...)
This way, when
NeuralNetwill take care of setting the
momentumparameter to 0.95.
(Note that the double underscore notation in
optimizer__momentummeans that the parameter
momentumshould be set on the object
optimizer. This is the same semantic as used by sklearn.)
Furthermore, this allows to change those parameters later:
This can be useful when you want to change certain parameters using a callback, when using the net in an sklearn grid search, etc.
- module : torch module (class or instance)
Module. In general, the uninstantiated class should be passed, although instantiated modules will also work.
- criterion : torch criterion (class, default=torch.nn.MSELoss)
Mean squared error loss.
- optimizer : torch optim (class, default=torch.optim.SGD)
The uninitialized optimizer (update rule) used to optimize the module
- lr : float (default=0.01)
Learning rate passed to the optimizer. You may use
lrinstead of using
optimizer__lr, which would result in the same outcome.
- max_epochs : int (default=10)
The number of epochs to train for each
fitcall. Note that you may keyboard-interrupt training at any time.
- batch_size : int (default=128)
Mini-batch size. Use this instead of setting
iterator_test__batch_size, which would result in the same outcome. If
batch_sizeis -1, a single batch with all the data will be used during training and validation.
- iterator_train : torch DataLoader
The default PyTorch
DataLoaderused for training data.
- iterator_valid : torch DataLoader
The default PyTorch
DataLoaderused for validation and test data, i.e. during inference.
- dataset : torch Dataset (default=skorch.dataset.Dataset)
The dataset is necessary for the incoming data to work with pytorch’s
DataLoader. It has to implement the
__getitem__methods. The provided dataset should be capable of dealing with a lot of data types out of the box, so only change this if your data is not supported. You should generally pass the uninitialized
Datasetclass and define additional arguments to X and y by prefixing them with
dataset__. It is also possible to pass an initialzed
Dataset, in which case no additional arguments may be passed.
- train_split : None or callable (default=skorch.dataset.ValidSplit(5))
None, there is no train/validation split. Else,
train_splitshould be a function or callable that is called with X and y data and should return the tuple
dataset_train, dataset_valid. The validation data may be
- callbacks : None, “disable”, or list of Callback instances (default=None)
Which callbacks to enable. There are three possible values:
callbacks=None, only use default callbacks, those returned by
callbacks="disable", disable all callbacks, i.e. do not run any of the callbacks, not even the default callbacks.
callbacksis a list of callbacks, use those callbacks in addition to the default callbacks. Each callback should be an instance of
Callback names are inferred from the class name. Name conflicts are resolved by appending a count suffix starting with 1, e.g.
EpochScoring_1. Alternatively, a tuple
(name, callback)can be passed, where
nameshould be unique. Callbacks may or may not be instantiated. The callback name can be used to set parameters on specific callbacks (e.g., for the callback with name
- predict_nonlinearity : callable, None, or ‘auto’ (default=’auto’)
The nonlinearity to be applied to the prediction. When set to ‘auto’, infers the correct nonlinearity based on the criterion (softmax for
CrossEntropyLossand sigmoid for
BCEWithLogitsLoss). If it cannot be inferred or if the parameter is None, just use the identity function. Don’t pass a lambda function if you want the net to be pickleable.
In case a callable is passed, it should accept the output of the module (the first output if there is more than one), which is a PyTorch tensor, and return the transformed PyTorch tensor.
This can be useful, e.g., when
predict_proba()should return probabilities but a criterion is used that does not expect probabilities. In that case, the module can return whatever is required by the criterion and the
predict_nonlinearitytransforms this output into probabilities.
- warm_start : bool (default=False)
Whether each fit call should lead to a re-initialization of the module (cold start) or whether the module should be trained further (warm start).
- verbose : int (default=1)
This parameter controls how much print output is generated by the net and its callbacks. By setting this value to 0, e.g. the summary scores at the end of each epoch are no longer printed. This can be useful when running a hyperparameter search. The summary scores are always logged in the history attribute, regardless of the verbose setting.
- device : str, torch.device, or None (default=’cpu’)
The compute device to be used. If set to ‘cuda’ in order to use GPU acceleration, data in torch tensors will be pushed to cuda tensors before being sent to the module. If set to None, then all compute devices will be left unmodified.
- compile : bool (default=False)
If set to
True, compile all modules using
torch.compile. For this to work, the installed torch version has to support
torch.compile. Compiled modules should work identically to non-compiled modules but should run faster on new GPU architectures (Volta and Ampere for instance). Additional arguments for
torch.compilecan be passed using the dunder notation, e.g. when initializing the net with
torch.compilewill be called with
- use_caching : bool or ‘auto’ (default=’auto’)
Optionally override the caching behavior of scoring callbacks. Callbacks such as
BatchScoringallow to cache the inference call to save time when calculating scores during training at the expense of memory. In certain situations, e.g. when memory is tight, you may want to disable caching. As it is cumbersome to change the setting on each callback individually, this parameter allows to override their behavior globally. By default (
'auto'), the callbacks will determine if caching is used or not. If this argument is set to
False, caching will be disabled on all callbacks. If set to
True, caching will be enabled on all callbacks. Implementation note: It is the job of the callbacks to honor this setting.
- prefixes_ : list of str
Contains the prefixes to special parameters. E.g., since there is the
'optimizer'prefix, it is possible to set parameters like so:
NeuralNet(..., optimizer__momentum=0.95). Some prefixes are populated dynamically, based on what modules and criteria are defined.
- cuda_dependent_attributes_ : list of str
Contains a list of all attribute prefixes whose values depend on a CUDA device. If a
NeuralNettrained with a CUDA-enabled device is unpickled on a machine without CUDA or with CUDA disabled, the listed attributes are mapped to CPU. Expand this list if you want to add other cuda-dependent attributes.
- initialized_ : bool
- module_ : torch module (instance)
The instantiated module.
- criterion_ : torch criterion (instance)
The instantiated criterion.
- callbacks_ : list of tuples
The complete (i.e. default and other), initialized callbacks, in a tuple with unique names.
- _modules : list of str
List of names of all modules that are torch modules. This list is collected dynamically when the net is initialized. Typically, there is no reason for a user to modify this list.
- _criteria : list of str
List of names of all criteria that are torch modules. This list is collected dynamically when the net is initialized. Typically, there is no reason for a user to modify this list.
- _optimizers : list of str
List of names of all optimizers. This list is collected dynamically when the net is initialized. Typically, there is no reason for a user to modify this list.
Checks whether the net is initialized
Check that the net is ready to train
Perform a forward step to produce the output used for prediction and scoring.
fit(X, y, **fit_params)
fit_loop(X[, y, epochs])
The proper fit loop.
forward(X[, training, device])
Gather and concatenate the output from forward call with input data.
forward_iter(X[, training, device])
Yield outputs of module forward calls on each batch of data.
Yield the learnable parameters of all modules
Get a dataset that contains the input data and is passed to the iterator.
Get an iterator that allows to loop over the batches of the given data.
get_loss(y_pred, y_true[, X, training])
Return the loss for this batch.
Collect and return init parameters for an attribute.
Collect and return init parameters for an optimizer.
Get internal train and validation datasets.
Return the train step accumulator.
Perform a single inference step on a batch of data.
Initializes all of its components and returns self.
Initializes all callbacks and save the result in the
Initializes the criterion.
Initializes the history.
Initializes the module.
Initialize the model optimizer.
Return an instance initialized with the given parameters
load_params([f_params, f_optimizer, …])
Loads the the module’s parameters, history, and optimizer, not the whole object.
Call the callback method specified in
method_namewith parameters specified in
on_batch_begin(net[, batch, training])
on_epoch_begin(net[, dataset_train, …])
on_epoch_end(net[, dataset_train, dataset_valid])
on_train_begin(net[, X, y])
on_train_end(net[, X, y])
partial_fit(X[, y, classes])
Fit the module.
Where applicable, return class labels for samples in X.
Return the output of the module’s forward method as a numpy array.
run_single_epoch(iterator, training, prefix, …)
Compute a single epoch of train or validation.
save_params([f_params, f_optimizer, …])
Saves the module’s parameters, history, and optimizer, not the whole object.
score(X, y[, sample_weight])
Return the coefficient of determination of the prediction.
Set the parameters of this class.
Compile torch modules
Prepares a loss function callable and pass it to the optimizer, hence performing one optimization step.
Compute y_pred, loss value, and update net’s gradients.
Remove all attributes not required for prediction.
Perform a forward step using batched data and return the resulting loss. get_default_callbacks get_params initialize_virtual_params on_batch_end on_grad_computed
fit(X, y, **fit_params)¶
In contrast to
yis non-optional to avoid mistakenly forgetting about
ycan be set to
Nonein case it is derived dynamically from