textformer.core¶
The core is the core. Essentially, it is the parent of everything. You should find parent classes defining the basis of our structure. They should provide variables and methods that will help to construct other modules.
A core package, containing all the basic class and functions that serves as the foundation of textformer common modules.
-
class
textformer.core.
Decoder
¶ Bases:
torch.nn.Module
A Decoder class is responsible for easily-implementing the decoding part of a neural network, when custom training or additional sets are not needed.
-
__init__
(self)¶ Initialization method.
Note that basic variables shared by all childs should be declared here, e.g., layers.
-
abstract
forward
(self, x)¶ Method that holds vital information whenever this class is called.
Note that you will need to implement this method directly on its child. Essentially, each neural network has its own forward pass implementation.
- Parameters
x (torch.Tensor) – A tensorflow’s tensor holding input data.
- Raises
NotImplementedError –
-
-
class
textformer.core.
Encoder
¶ Bases:
torch.nn.Module
An Encoder class is responsible for easily-implementing the encoding part of a neural network, when custom training or additional sets are not needed.
-
__init__
(self)¶ Initialization method.
Note that basic variables shared by all childs should be declared here, e.g., layers.
-
abstract
forward
(self, x)¶ Method that holds vital information whenever this class is called.
Note that you will need to implement this method directly on its child. Essentially, each neural network has its own forward pass implementation.
- Parameters
x (torch.Tensor) – A tensorflow’s tensor holding input data.
- Raises
NotImplementedError –
-
-
class
textformer.core.
Model
(encoder, decoder, ignore_token=None, init_weights=None, device='cpu')¶ Bases:
torch.nn.Module
A Model class is responsible for customly implementing Sequence-To-Sequence and Transformer architectures.
One can configure, if necessary, different properties or methods that can be used throughout all childs.
-
property
D
(self)¶ Decoder: Decoder architecture.
-
property
E
(self)¶ Encoder: Encoder architecture.
-
__init__
(self, encoder, decoder, ignore_token=None, init_weights=None, device='cpu')¶ Initialization method.
- Parameters
encoder (Encoder) – Network’s encoder architecture.
decoder (Decoder) – Network’s decoder architecture.
init_weights (tuple) – Tuple holding the minimum and maximum values for weights initialization.
ignore_token (int) – The index of a token to be ignore by the loss function.
device (str) – Device that model should be trained on, e.g., cpu or cuda.
-
_compile
(self, ignore_token, init_weights)¶ Compiles the network by setting its optimizer, loss function and additional properties.
-
abstract
bleu
(self, dataset, src_field, trg_field, max_length=50, n_grams=4)¶ Calculates BLEU score over a dataset from its difference between targets and predictions.
Note that you will need to implement this method directly on its child. Essentially, each neural network has its own bleu implementation, due to having different translation methods.
- Parameters
dataset (torchtext.data.Dataset) – Dataset to have its BLEU calculated.
src_field (torchtext.data.Field) – Source vocabulary datatype instructions for tensor convertion.
trg_field (torchtext.data.Field) – Target vocabulary datatype instructions for tensor convertion.
max_length (int) – Maximum length of translated text.
n_grams (int) – Maxmimum n-grams to be used.
- Raises
NotImplementedError –
-
property
device
(self)¶ str: Indicates which device is being used for computation.
-
dump
(self, **kwargs)¶ Dumps any amount of keyword documents to lists in the history property.
-
evaluate
(self, test_iterator)¶ Evaluates the model.
- Parameters
test_iterator (torchtext.data.Iterator) – Testing data iterator.
-
fit
(self, train_iterator, val_iterator=None, epochs=10)¶ Trains the model.
- Parameters
train_iterator (torchtext.data.Iterator) – Training data iterator.
val_iterator (torchtext.data.Iterator) – Validation data iterator.
epochs (int) – The maximum number of training epochs.
-
abstract
generate_text
(self, start, field, length=10, temperature=1.0)¶ Generates text by feeding to the network the current token (t) and predicting the next token (t+1).
Note that you will need to implement this method directly on its child. Essentially, each neural network has its own sample (text generation) implementation.
- Parameters
start (str) – The start string to generate the text.
field (torchtext.data.Field) – Datatype instructions for tensor convertion.
length (int) – Length of generated text.
temperature (float) – A temperature value to sample the token.
- Raises
NotImplementedError –
-
property
history
(self)¶ dict: Dictionary containing historical values from the model.
-
step
(self, batch, clip)¶ Performs a single batch optimization step.
- Parameters
batch (tuple) – Tuple containing the batches input (x) and target (y).
clip (float) – Value to clip the gradients.
- Returns
The training loss accross the batch.
-
abstract
translate_text
(self, start, src_field, trg_field, max_length=10)¶ Translates text from the source vocabulary to the target vocabulary.
Note that you will need to implement this method directly on its child. Essentially, each neural network has its own translation implementation.
- Parameters
start (str) – The string to be translated.
src_field (torchtext.data.Field) – Source vocabulary datatype instructions for tensor convertion.
trg_field (torchtext.data.Field) – Target vocabulary datatype instructions for tensor convertion.
max_length (int) – Maximum length of translated text.
- Raises
NotImplementedError –
-
val_step
(self, batch)¶ Performs a single batch evaluation step.
- Parameters
batch (tuple) – Tuple containing the batches input (x) and target (y).
- Returns
The validation loss accross the batch.
-
property