pydgn.training

training.callback

• callback.early_stopping
• callback.engine_callback
• callback.gradient_clipping
• callback.metric
• callback.optimizer
• callback.plotter
• callback.scheduler

training.event

• event.dispatcher
• event.handler
• event.state

training.engine

class pydgn.training.engine.DataStreamTrainingEngine(engine_callback: Callable[[...], pydgn.training.callback.engine_callback.EngineCallback], model: pydgn.model.interface.ModelInterface, loss: pydgn.training.callback.metric.Metric, optimizer: pydgn.training.callback.optimizer.Optimizer, scorer: pydgn.training.callback.metric.Metric, scheduler: Optional[pydgn.training.callback.scheduler.Scheduler] = None, early_stopper: Optional[pydgn.training.callback.early_stopping.EarlyStopper] = None, gradient_clipper: Optional[pydgn.training.callback.gradient_clipping.GradientClipper] = None, device: str = 'cpu', plotter: Optional[pydgn.training.callback.plotter.Plotter] = None, exp_path: Optional[str] = None, evaluate_every: int = 1, store_last_checkpoint: bool = False, reset_eval_model_hidden_state: bool = True)

Bases: pydgn.training.engine.TrainingEngine

Class that handles a stream of graphs that could end at any moment.

_loop(loader: torch_geometric.loader.DataLoader)

Compared to superclass version, handles the issue of a stream of data that could end at any moment. This is done using an additional boolean state variable.

class pydgn.training.engine.GraphSequenceTrainingEngine(engine_callback: Callable[[...], pydgn.training.callback.engine_callback.EngineCallback], model: pydgn.model.interface.ModelInterface, loss: pydgn.training.callback.metric.Metric, optimizer: pydgn.training.callback.optimizer.Optimizer, scorer: pydgn.training.callback.metric.Metric, scheduler: Optional[pydgn.training.callback.scheduler.Scheduler] = None, early_stopper: Optional[pydgn.training.callback.early_stopping.EarlyStopper] = None, gradient_clipper: Optional[pydgn.training.callback.gradient_clipping.GradientClipper] = None, device: str = 'cpu', plotter: Optional[pydgn.training.callback.plotter.Plotter] = None, exp_path: Optional[str] = None, evaluate_every: int = 1, store_last_checkpoint: bool = False, reset_eval_model_hidden_state: bool = True)

Bases: pydgn.training.engine.TrainingEngine

Assumes that the model can return None predictions (i.e., output[0] below) whenever no values should be predicted according to the mask field in the Data object representing a snapshot

_loop_helper()

Compared to superclass version, aas the batch is composed of a list of snapshots, where each snapshot is a graph object, we add an inner loop to pass one snapshot at a time.

class pydgn.training.engine.LinkPredictionSingleGraphEngine(engine_callback: Callable[[...], pydgn.training.callback.engine_callback.EngineCallback], model: pydgn.model.interface.ModelInterface, loss: pydgn.training.callback.metric.Metric, optimizer: pydgn.training.callback.optimizer.Optimizer, scorer: pydgn.training.callback.metric.Metric, scheduler: Optional[pydgn.training.callback.scheduler.Scheduler] = None, early_stopper: Optional[pydgn.training.callback.early_stopping.EarlyStopper] = None, gradient_clipper: Optional[pydgn.training.callback.gradient_clipping.GradientClipper] = None, device: str = 'cpu', plotter: Optional[pydgn.training.callback.plotter.Plotter] = None, exp_path: Optional[str] = None, evaluate_every: int = 1, store_last_checkpoint: bool = False, reset_eval_model_hidden_state: bool = True)

Bases: pydgn.training.engine.TrainingEngine

Specific engine for link prediction tasks. Here, we expect target values in the form of tuples: (_, pos_edges, neg_edges), where pos_edges and neg_edges have been generated by the splitter and provided by the data provider.

_num_targets(targets)

Computes the number of targets as the positive links + negative links

_to_data_list(x, batch, y)

Converts model outputs back to a list of Data elements. Useful for incremental architectures.

Parameters

• x (torch.Tensor) – tensor holding information of different nodes/graphs embeddings

• batch (torch.Tensor) – the usual PyG batch tensor. Used to split node/graph embeddings graph-wise.

• y (torch.Tensor) – target labels, used to determine whether the task is graph prediction or node prediction. Can be None.

Returns

a list of PyG Data objects (with only x and y attributes)

_to_list(data_list, embeddings, batch, edge_index, y)

Extends the data_list list of PyG Data objects with new samples.

Parameters

• data_list – a list of PyG Data objects (with only x and y attributes)

• embeddings (torch.Tensor) – tensor holding information of different nodes/graphs embeddings

• batch (torch.Tensor) – the usual PyG batch tensor. Used to split node/graph embeddings graph-wise.

• edge_index –

• y (torch.Tensor) – target labels, used to determine whether the task is graph prediction or node prediction. Can be None.

Returns

a list of PyG Data objects (with only x and y attributes)

class pydgn.training.engine.TrainingEngine(engine_callback: Callable[[...], pydgn.training.callback.engine_callback.EngineCallback], model: pydgn.model.interface.ModelInterface, loss: pydgn.training.callback.metric.Metric, optimizer: pydgn.training.callback.optimizer.Optimizer, scorer: pydgn.training.callback.metric.Metric, scheduler: Optional[pydgn.training.callback.scheduler.Scheduler] = None, early_stopper: Optional[pydgn.training.callback.early_stopping.EarlyStopper] = None, gradient_clipper: Optional[pydgn.training.callback.gradient_clipping.GradientClipper] = None, device: str = 'cpu', plotter: Optional[pydgn.training.callback.plotter.Plotter] = None, exp_path: Optional[str] = None, evaluate_every: int = 1, store_last_checkpoint: bool = False, reset_eval_model_hidden_state: bool = True)

Bases: pydgn.training.event.dispatcher.EventDispatcher

This is the most important class when it comes to training a model. It implements the EventDispatcher interface, which means that after registering some callbacks in a given order, it will proceed to trigger specific events that will result in the shared State object being updated by the callbacks. Callbacks implement the EventHandler interface, and they receive the shared State object when any event is triggered. Knowing the order in which callbacks are called is important. The order is:

• loss function

• score function

• gradient clipper

• optimizer

• early stopper

• scheduler

• plotter

Parameters

• engine_callback – (Callable[…, EngineCallback]): the engine callback object to be used for data fetching and checkpoints (or even other purposes if necessary)

• model (ModelInterface) – the model to be trained

• loss (Metric) – the loss to be used

• optimizer (Optimizer) – the optimizer to be used

• scorer (Metric) – the score to be used

• scheduler (Scheduler) – the scheduler to be used Default is None.

• early_stopper –

  (EarlyStopper):

  the early stopper to be used. Default is None.

• gradient_clipper – (GradientClipper ): the gradient clipper to be used. Default is None.

• device (str) – the device on which to train. Default is cpu.

• plotter (Plotter) – the plotter to be used. Default is None.

• exp_path (str) – the path of the experiment folder. Default is None but it is always instantiated.

• evaluate_every (int) – the frequency of logging epoch results. Default is 1.

• store_last_checkpoint (bool) – whether to store a checkpoint at the end of each epoch. Allows to resume training from last epoch. Default is False.

• reset_eval_model_hidden_state (bool) – [temporal graph learning] Used when we want to reset the state after performing previous inference. It should be False when we are dealing with a single temporal graph sequence, because we don’t want to reset the hidden state after processing the previous [training/validation] time steps.

_loop(loader: torch_geometric.loader.DataLoader)

Main method that computes a pass over the dataset using the data loader provided.

Parameters

loader (torch_geometric.loader.DataLoader) – the loader to be used

_loop_helper()

Helper function that loops over the data.

_num_targets(targets: torch.Tensor) → int

Computes the number of targets (different from the dimension of each target).

:param targets (torch.Tensor: the ground truth tensor

Returns

an integer with the number of targets to predict

_restore_checkpoint_and_best_results(ckpt_filename, best_ckpt_filename, zero_epoch)

Restores the (best or last) checkpoint from a given file, and loads the best results so far into the state if any.

_to_data_list(x: torch.Tensor, batch: torch.Tensor, y: Optional[torch.Tensor]) → List[torch_geometric.data.Data]

Converts model outputs back to a list of Data elements. Useful for incremental architectures.

Parameters

• x (torch.Tensor) – tensor holding information of different nodes/graphs embeddings

• batch (torch.Tensor) – the usual PyG batch tensor. Used to split node/graph embeddings graph-wise.

• y (torch.Tensor) – target labels, used to determine whether the task is graph prediction or node prediction. Can be None.

Returns

a list of PyG Data objects (with only x and y attributes)

_to_list(data_list: List[torch_geometric.data.Data], embeddings: Union[Tuple[torch.Tensor], torch.Tensor], batch: torch.Tensor, edge_index: torch.Tensor, y: Optional[torch.Tensor]) → List[torch_geometric.data.Data]

Extends the data_list list of PyG Data objects with new samples.

Parameters

• data_list – a list of PyG Data objects (with only x and y attributes)

• embeddings (torch.Tensor) – tensor holding information of different nodes/graphs embeddings

• batch (torch.Tensor) – the usual PyG batch tensor. Used to split node/graph embeddings graph-wise.

• edge_index –

• y (torch.Tensor) – target labels, used to determine whether the task is graph prediction or node prediction. Can be None.

Returns

a list of PyG Data objects (with only x and y attributes)

_train(loader)

Implements a loop over the data in training mode

infer(loader: torch_geometric.loader.DataLoader, set: str) → Tuple[dict, dict, List[torch_geometric.data.Data]]

Performs an evaluation step on the data.

Parameters

• loader (torch_geometric.loader.DataLoader) – the loader to be used

• set (str) – the type of dataset being used, can be TRAINING, VALIDATION or TEST (as defined in pydgn.static)

Returns

a tuple (loss dict, score dict, list of torch_geometric.data.Data objects with x and y attributes only). The data list can be used, for instance, in semi-supervised experiments or in incremental architectures

set_device()

Moves the model and the loss metric to the proper device.

set_eval_mode()

Sets the model and the internal state in EVALUATION mode

set_training_mode()

Sets the model and the internal state in TRAINING mode

train(train_loader: torch_geometric.loader.DataLoader, validation_loader: Optional[torch_geometric.loader.DataLoader] = None, test_loader: Optional[torch_geometric.loader.DataLoader] = None, max_epochs: int = 100, zero_epoch: bool = False, logger: Optional[pydgn.log.logger.Logger] = None) → Tuple[dict, dict, List[torch_geometric.data.Data], dict, dict, List[torch_geometric.data.Data], dict, dict, List[torch_geometric.data.Data]]

Trains the model and regularly evaluates on validation and test data (if given). May perform early stopping and checkpointing.

Parameters

• train_loader (torch_geometric.loader.DataLoader) – the DataLoader associated with training data

• validation_loader (torch_geometric.loader.DataLoader) – the DataLoader associated with validation data, if any

• test_loader (torch_geometric.loader.DataLoader) – the DataLoader associated with test data, if any

• max_epochs (int) – maximum number of training epochs. Default is 100

• zero_epoch – if True, starts again from epoch 0 and resets optimizer and scheduler states. Default is False

• logger – the logger

Returns

a tuple (train_loss, train_score, train_embeddings, validation_loss, validation_score, validation_embeddings, test_loss, test_score, test_embeddings)

pydgn.training.engine.log(msg, logger: pydgn.log.logger.Logger)

Logs a message using a logger

pydgn.training.engine.reorder(obj: List[object], perm: List[int])

Reorders a list of objects in ascending order according to the indices defined in permutation argument.

Parameters

• obj (List[object]) – the list of objects

• perm (List[int]) – the permutation

Returns

The reordered list of objects

training.profiler

class pydgn.training.profiler.Profiler(threshold: float)

Bases: object

A decorator class that is applied to a EventHandler object implementing a set of callback functions. For each callback, the Profiler stores the average and total running time across epochs. When the experiment terminates (either correctly or abruptly) the Profiler can produce a report to be stored in the experiment’s log file.

The Profiler is used as a singleton, and it produces wrappers that update its own state.

Parameters

threshold (float) – used to filter out callback functions that consume a negligible amount of time from the report

Usage:

Istantiate a profiler, and then register an event_handler with the syntax profiler(event_handler), which returns another object implementing the EventHandler interface

report() → str

Builds a report string containing the statistics of the experiment accumulated so far.

Returns

a string containing the report

training.util

pydgn.training.util.atomic_save(data: dict, filepath: str)

Atomically stores a dictionary that can be serialized by torch.save(), exploiting the atomic os.replace().

Parameters

• data (dict) – the dictionary to be stored

• filepath (str) – the absolute filepath where to store the dictionary