deletor.models package

Submodules

deletor.models.attn module

class deletor.models.attn.AttentionLayer(*args, **kwargs)

Bases: tensorflow.python.keras.engine.base_layer.Layer

Parameters

• n_model –

• n_ff_units –

• use_layer_norm –

• dropout_rate –

call(x, training, mask, **kwargs)

Parameters

• x –

• training –

• mask –

• kwargs –

Returns

class deletor.models.attn.Encoder(*args, **kwargs)

Bases: tensorflow.python.keras.engine.base_layer.Layer

Parameters

• num_layers –

• n_model –

• num_heads –

• d_ff –

• dropout_rate –

call(x, mask, training, **kwargs)

Parameters

• x –

• mask –

• training –

• kwargs –

Returns

class deletor.models.attn.EncoderLayer(*args, **kwargs)

Bases: tensorflow.python.keras.engine.base_layer.Layer

Parameters

• n_model –

• num_heads –

• d_ff –

• dropout_rate –

call(x, training, mask, **kwargs)

Parameters

• x –

• training –

• mask –

• kwargs –

Returns

class deletor.models.attn.GroupwiseAttentionNetwork(*args, **kwargs)

Bases: tensorflow.python.keras.engine.training.Model

Parameters

• params –

• kwargs –

call(x: Dict[str, tensorflow.python.framework.ops.Tensor], training: bool = True, **kwargs)

Parameters

• x –

• training –

• kwargs –

Returns

classmethod make_shared_layer(n_model: int)

Parameters

n_model –

Returns

class deletor.models.attn.GroupwiseMultiHeadAttentionNetwork(*args, **kwargs)

Bases: tensorflow.python.keras.engine.training.Model

This model implements a groupwise scoring model using normalized dot product attention mechanism based on Attention Is All You Need.

Parameters

• params – See ModelParameter for the valid parameters.

• kwargs –

call(x, training: bool = True, **kwargs)

Parameters

• x –

• training –

• kwargs –

Returns

default_share_weights = False

default_use_average = True

class deletor.models.attn.ModelParameter

Bases: object

DROPOUT_RATE = 'dropout_rate'

GROUP_SIZE = 'group_size'

N_FEATURES = 'n_features'

N_FF_UNITS = 'n_feed_forward_units'

N_HEADS = 'n_heads'

N_LAYERS = 'n_layers'

N_MODEL = 'n_model'

SHARE_WEIGHTS = 'share_weights'

USE_AVERAGE = 'use_average'

USE_LAYER_NORM = 'use_layer_norm'

class deletor.models.attn.MultiHeadAttention(*args, **kwargs)

Bases: tensorflow.python.keras.engine.base_layer.Layer

Parameters

• n_model –

• num_heads –

call(v, k, q, mask, **kwargs)

Parameters

• v –

• k –

• q –

• mask –

• kwargs –

Returns

classmethod make_attention_mask(x, y, multi_head: bool = True, pad_value: float = - 3.4028235e+38)

Parameters

• x –

• y –

• multi_head –

• pad_value –

Returns

classmethod reshape_input(q: tensorflow.python.framework.ops.Tensor, k: tensorflow.python.framework.ops.Tensor, v: tensorflow.python.framework.ops.Tensor)

classmethod reshape_output(output: tensorflow.python.framework.ops.Tensor, q: tensorflow.python.framework.ops.Tensor)

classmethod scaled_dot_product_attention(q: tensorflow.python.framework.ops.Tensor, k: tensorflow.python.framework.ops.Tensor, v: tensorflow.python.framework.ops.Tensor, mask: tensorflow.python.framework.ops.Tensor = None)

Calculate the attention weights.

q, k, v must have matching leading dimensions. k, v must have matching penultimate dimension, i.e.: seq_len_k = seq_len_v. The mask has different shapes depending on its type(padding or look ahead) but it must be broadcastable for addition.

Parameters

• q – query shape == (…, seq_len_q, depth)

• k – key shape == (…, seq_len_k, depth)

• v – value shape == (…, seq_len_v, depth_v)

• mask – Float tensor with shape broadcastable to (…, seq_len_q, seq_len_k). Defaults to None.

Returns

output, attention_weights

split_heads(x, batch_size)

Split the last dimension into (num_heads, depth). Transpose the result such that the shape is (batch_size, num_heads, seq_len, depth)

class deletor.models.attn.SelfAttention(*args, **kwargs)

Bases: tensorflow.python.keras.engine.base_layer.Layer

Parameters

• n_layers –

• n_model –

• n_ff_units –

• use_layer_norm –

• dropout_rate –

call(x, mask, training, **kwargs)

Parameters

• x –

• mask –

• training –

• kwargs –

Returns

deletor.models.attn.make_point_wise_feed_forward_network(n_model, dff)

Parameters

• n_model –

• dff –

Returns

deletor.models.boost module

class deletor.models.boost.EnsembleScoringNetwork(*args, **kwargs)

Bases: deletor.models.gsf.GroupwiseScoringNetwork

Parameters

• params – See ModelParameter for the valid parameters.

• index_pad_value –

• kwargs –

apply_weak_models(x, y)

class deletor.models.boost.ModelParameter

Bases: deletor.models.gsf.ModelParameter

N_SAMPLED_FEATURES = 'n_sampled_features'

class deletor.models.boost.WeakGroupwiseScoringNetwork(*args, **kwargs)

Bases: deletor.models.gsf.GroupwiseScoringNetwork

Parameters

• params – See ModelParameter for the valid parameters.

• index_pad_value –

• kwargs –

deletor.models.gsf module

Documentation

class deletor.models.gsf.GroupwiseInputNetwork(*args, **kwargs)

Bases: tensorflow.python.keras.engine.training.Model

Parameters

• params – See ModelParameter for the valid parameters.

• index_pad_value –

• kwargs –

call(inputs, training: bool = True, **kwargs)

default_group_size = 16

default_share_weights = False

default_use_average = True

class deletor.models.gsf.GroupwiseScoringNetwork(*args, **kwargs)

Bases: tensorflow.python.keras.engine.training.Model

This class (tries to) implements the GSF model from Ai et al.

Parameters

• params – See ModelParameter for the valid parameters.

• index_pad_value –

• kwargs –

call(inputs, training: bool = True, **kwargs)

Parameters

• inputs –

• training –

Returns

default_group_size = 16

default_share_weights = False

default_use_average = True

class deletor.models.gsf.GroupwiseScoringNetwork2(*args, **kwargs)

Bases: tensorflow.python.keras.engine.training.Model

call(inputs, training: bool = True, **kwargs)

Parameters

• inputs –

• training –

Returns

default_group_size = 16

default_share_weights = False

default_use_average = True

class deletor.models.gsf.ModelParameter

Bases: object

DROPOUT_RATE = 'dropout_rate'

(Optional) The dropout rate for each layer. This can be a single number in which case the same dropout will be applied for each layer. Or it can be a list of the same size as N_UNITS. (Default: 0)

GROUP_SIZE = 'group_size'

(Required) The number of documents in each group.

N_FEATURES = 'n_features'

(Required) The number of features in the dataset (e.g., 136 for MSLR)

N_UNITS = 'n_units'

(Required) The number of hidden units for each layer.

RANDOM_SEED = 'random_seed'

(Optional) A random seed to use for any stochastic operations. (Default: `None`)

SHARE_WEIGHTS = 'share_weights'

(Optional) If `True` then each document in the input is first passed through a dense layer before being concatenated into groups. (Default: `False`)

USE_AVERAGE = 'use_average'

(Optional) If `True` the final output will be an average over each document across samples, otherwise it will be the sum. (Default: `True`)

deletor.models.mlp module

Documentation

class deletor.models.mlp.ModelParameter

Bases: object

DROPOUT_RATE = 'dropout_rate'

(Optional) The dropout rate for each layer. This can be a single number in which case the same dropout will be applied for each layer. Or it can be a list of the same size as `N_UNITS`.

LIST_SIZE = 'list_size'

(Optional) The maximum number of documents to include for each query or `None` for all of them.

N_UNITS = 'n_units'

(Required) The number of hidden units for each layer.

RANDOM_SEED = 'random_seed'

(Optional) A random seed to use for any stochastic operations.

class deletor.models.mlp.SimpleScoringNetwork(*args, **kwargs)

Bases: tensorflow.python.keras.engine.training.Model

A baseline model that uses a fully connected dense network, with one output, for ranking.

Parameters

• params – See ModelParameter for the valid parameters.

• kwargs –

call(inputs, training: bool = True, **kwargs)

Parameters

• inputs –

• training –

• kwargs –

Returns

default_dropout_rate = 0.0

default_share_weights = False

deletor.models.utils module

Documentation

deletor.models.utils.normalize_dropout(dropout_rate: Optional[Union[List[float], Tuple[float], float]], n_units: List[int])

Returns the dropout rate as a list the same size as len(n_units). This does some error checking, but is not very robust.

Parameters

• dropout_rate –

• n_units –

Returns

Module contents

Documentation