textformer.core

The core is the core. Essentially, it is the parent of everything. You should find parent classes defining the basis of our structure. They should provide variables and methods that will help to construct other modules.

A core package, containing all the basic class and functions that serves as the foundation of textformer common modules.

class textformer.core.Decoder

Bases: torch.nn.Module

A Decoder class is responsible for easily-implementing the decoding part of a neural network, when custom training or additional sets are not needed.

__init__(self)

Initialization method.

Note that basic variables shared by all childs should be declared here, e.g., layers.

abstract forward(self, x)

Method that holds vital information whenever this class is called.

Note that you will need to implement this method directly on its child. Essentially, each neural network has its own forward pass implementation.

Parameters

x (torch.Tensor) – A tensorflow’s tensor holding input data.

Raises

NotImplementedError

class textformer.core.Encoder

Bases: torch.nn.Module

An Encoder class is responsible for easily-implementing the encoding part of a neural network, when custom training or additional sets are not needed.

__init__(self)

Initialization method.

Note that basic variables shared by all childs should be declared here, e.g., layers.

abstract forward(self, x)

Method that holds vital information whenever this class is called.

Note that you will need to implement this method directly on its child. Essentially, each neural network has its own forward pass implementation.

Parameters

x (torch.Tensor) – A tensorflow’s tensor holding input data.

Raises

NotImplementedError

class textformer.core.Model(encoder, decoder, ignore_token=None, init_weights=None, device='cpu')

Bases: torch.nn.Module

A Model class is responsible for customly implementing Sequence-To-Sequence and Transformer architectures.

One can configure, if necessary, different properties or methods that can be used throughout all childs.

property D(self)

Decoder: Decoder architecture.

property E(self)

Encoder: Encoder architecture.

__init__(self, encoder, decoder, ignore_token=None, init_weights=None, device='cpu')

Initialization method.

Parameters
  • encoder (Encoder) – Network’s encoder architecture.

  • decoder (Decoder) – Network’s decoder architecture.

  • init_weights (tuple) – Tuple holding the minimum and maximum values for weights initialization.

  • ignore_token (int) – The index of a token to be ignore by the loss function.

  • device (str) – Device that model should be trained on, e.g., cpu or cuda.

_compile(self, ignore_token, init_weights)

Compiles the network by setting its optimizer, loss function and additional properties.

abstract bleu(self, dataset, src_field, trg_field, max_length=50, n_grams=4)

Calculates BLEU score over a dataset from its difference between targets and predictions.

Note that you will need to implement this method directly on its child. Essentially, each neural network has its own bleu implementation, due to having different translation methods.

Parameters
  • dataset (torchtext.data.Dataset) – Dataset to have its BLEU calculated.

  • src_field (torchtext.data.Field) – Source vocabulary datatype instructions for tensor convertion.

  • trg_field (torchtext.data.Field) – Target vocabulary datatype instructions for tensor convertion.

  • max_length (int) – Maximum length of translated text.

  • n_grams (int) – Maxmimum n-grams to be used.

Raises

NotImplementedError

property device(self)

str: Indicates which device is being used for computation.

dump(self, **kwargs)

Dumps any amount of keyword documents to lists in the history property.

evaluate(self, test_iterator)

Evaluates the model.

Parameters

test_iterator (torchtext.data.Iterator) – Testing data iterator.

fit(self, train_iterator, val_iterator=None, epochs=10)

Trains the model.

Parameters
  • train_iterator (torchtext.data.Iterator) – Training data iterator.

  • val_iterator (torchtext.data.Iterator) – Validation data iterator.

  • epochs (int) – The maximum number of training epochs.

abstract generate_text(self, start, field, length=10, temperature=1.0)

Generates text by feeding to the network the current token (t) and predicting the next token (t+1).

Note that you will need to implement this method directly on its child. Essentially, each neural network has its own sample (text generation) implementation.

Parameters
  • start (str) – The start string to generate the text.

  • field (torchtext.data.Field) – Datatype instructions for tensor convertion.

  • length (int) – Length of generated text.

  • temperature (float) – A temperature value to sample the token.

Raises

NotImplementedError

property history(self)

dict: Dictionary containing historical values from the model.

step(self, batch, clip)

Performs a single batch optimization step.

Parameters
  • batch (tuple) – Tuple containing the batches input (x) and target (y).

  • clip (float) – Value to clip the gradients.

Returns

The training loss accross the batch.

abstract translate_text(self, start, src_field, trg_field, max_length=10)

Translates text from the source vocabulary to the target vocabulary.

Note that you will need to implement this method directly on its child. Essentially, each neural network has its own translation implementation.

Parameters
  • start (str) – The string to be translated.

  • src_field (torchtext.data.Field) – Source vocabulary datatype instructions for tensor convertion.

  • trg_field (torchtext.data.Field) – Target vocabulary datatype instructions for tensor convertion.

  • max_length (int) – Maximum length of translated text.

Raises

NotImplementedError

val_step(self, batch)

Performs a single batch evaluation step.

Parameters

batch (tuple) – Tuple containing the batches input (x) and target (y).

Returns

The validation loss accross the batch.