deletor package

Submodules

deletor.constants module

Documentation

deletor.losses module

class deletor.losses.ApproximateBiDiNormalizedDiscountedCumulativeGain(reduce: bool = True, max_label: float = 4.0, beta: float = 1.0, alpha: float = 10.0, pad_value: float = - 3.4028235e+38)[source]

Bases: deletor.losses.ListwiseLoss

Parameters

  • reduce –

  • max_label –

  • beta –

  • alpha –

  • pad_value –

compute(labels: tensorflow.python.framework.ops.Tensor, logits: tensorflow.python.framework.ops.Tensor, **kwargs)[source]
Parameters

  • labels –

  • logits –

  • kwargs –

Returns

compute_unreduced(labels: tensorflow.python.framework.ops.Tensor, logits: tensorflow.python.framework.ops.Tensor)[source]
Parameters

  • labels –

  • logits –

Returns

dcg(labels: tensorflow.python.framework.ops.Tensor, logits: tensorflow.python.framework.ops.Tensor)[source]
Parameters

  • labels –

  • logits –

Returns

class deletor.losses.ApproximateNormalizedDiscountedCumulativeGain(reduce: bool = True, lambda_weights=None, alpha: float = 10.0, pad_value: float = - 3.4028235e+38)[source]

Bases: deletor.losses.ListwiseLoss

Parameters

  • reduce –

  • lambda_weights –

  • alpha –

  • pad_value –

compute(labels: tensorflow.python.framework.ops.Tensor, logits: tensorflow.python.framework.ops.Tensor, **kwargs)[source]
Parameters

  • y_true –

  • y_pred –

  • kwargs –

Returns

compute_unreduced(labels: tensorflow.python.framework.ops.Tensor, logits: tensorflow.python.framework.ops.Tensor)[source]
name()[source]
class deletor.losses.DCGLambdaWeight(k: int = None, gain_fn: Callable = <function DCGLambdaWeight.<lambda>>, discount_fn: Callable = <function DCGLambdaWeight.<lambda>>, normalized: bool = False, smooth_fraction: float = 0.0)[source]

Bases: deletor.losses.LambdaWeight

LambdaWeight for Discounted Cumulative Gain metric.

Constructor.

Ranks are 1-based, not 0-based. Given rank i and j, there are two types of pair weights:

u = |rank_discount_fn(|i-j|) - rank_discount_fn(|i-j| + 1)|
v = |rank_discount_fn(i) - rank_discount_fn(j)|
where u is the newly introduced one in LambdaLoss paper
(https://ai.google/research/pubs/pub47258) and v is the original one in the
LambdaMART paper “From RankNet to LambdaRank to LambdaMART: An Overview”.
The final pair weight contribution of ranks is
(1-smooth_fraction) * u + smooth_fraction * v.
Parameters

  • k – The top k for the DCG metric.

  • gain_fn – Transforms labels.

  • discount_fn – The rank discount function.

  • normalized – If True, normalize weight by the max DCG.

  • smooth_fraction – parameter to control the contribution from LambdaMART.

discount_for_absolute_rank(ranks, k)[source]

Standard discount in the LambdaMART paper.

discount_for_relative_rank_diff(ranks, k)[source]

Rank-based discount in the LambdaLoss paper.

individual_weights(labels, ranks)[source]

Returns the weight Tensor for individual examples.

Parameters

  • labels – A dense Tensor of labels with shape [batch_size, list_size].

  • ranks – A dense Tensor of ranks with the same shape as labels that are sorted by logits.

Returns

A Tensor that can weight individual examples.

pair_weights(labels, ranks)[source]

Returns the weight adjustment Tensor for example pairs.

Parameters

  • labels – A dense Tensor of labels with shape [batch_size, list_size].

  • ranks – A dense Tensor of ranks with the same shape as labels that are sorted by logits.

Returns

A Tensor that can weight example pairs.

class deletor.losses.LambdaWeight[source]

Bases: abc.ABC

Interface for ranking metric optimization.

This class wraps weights used in the LambdaLoss framework for ranking metric optimization (https://ai.google/research/pubs/pub47258). Such an interface is to be instantiated by concrete lambda weight models. The instance is used together with standard loss such as logistic loss and softmax loss.

individual_weights(labels, ranks)[source]

Returns the weight Tensor for individual examples.

Parameters

  • labels – A dense Tensor of labels with shape [batch_size, list_size].

  • ranks – A dense Tensor of ranks with the same shape as labels that are sorted by logits.

Returns

A Tensor that can weight individual examples.

abstract pair_weights(labels, ranks)[source]

Returns the weight adjustment Tensor for example pairs.

Parameters

  • labels – A dense Tensor of labels with shape [batch_size, list_size].

  • ranks – A dense Tensor of ranks with the same shape as labels that are sorted by logits.

Returns

A Tensor that can weight example pairs.

class deletor.losses.ListwiseLoss(name: str, lambda_weight: deletor.losses.LambdaWeight = None, params: Dict = None)[source]

Bases: deletor.losses.NetworkLoss, abc.ABC

Parameters

  • name –

  • lambda_weight –

  • params –

property name
normalize_weights(labels, weights)[source]
Parameters

  • labels –

  • weights –

Returns

class deletor.losses.MeanSquaredError(reduce: bool = True, weight_by_labels: bool = True, pad_value=- 3.4028235e+38)[source]

Bases: deletor.losses.NetworkLoss

Parameters

  • reduce –

  • weight_by_labels –

  • pad_value –

compute(y_true: tensorflow.python.framework.ops.Tensor, y_pred: tensorflow.python.framework.ops.Tensor, **kwargs)[source]
Parameters

  • y_true –

  • y_pred –

  • kwargs –

Returns

class deletor.losses.MultiLoss(base_losses: Iterable[deletor.losses.NetworkLoss], weights: Optional[Union[float, List[float], numpy.ndarray]] = None, reduce: bool = True)[source]

Bases: deletor.losses.NetworkLoss

Parameters

  • base_losses –

  • weights –

  • reduce –

compute(y_true: tensorflow.python.framework.ops.Tensor, y_pred: tensorflow.python.framework.ops.Tensor, **kwargs)[source]
Parameters

  • y_true –

  • y_pred –

  • kwargs –

Returns

class deletor.losses.NetworkLoss(reduction='auto', name=None)[source]

Bases: tensorflow.python.keras.losses.Loss, abc.ABC

Base class for loss functions.

Initializes Loss class.

Args:
reduction: (Optional) Type of tf.keras.losses.Reduction to apply to

loss. Default value is AUTO. AUTO indicates that the reduction option will be determined by the usage context. For almost all cases this defaults to SUM_OVER_BATCH_SIZE. When used with tf.distribute.Strategy, outside of built-in training loops such as tf.keras compile and fit, using AUTO or SUM_OVER_BATCH_SIZE will raise an error. Please see this custom training [tutorial] (https://www.tensorflow.org/tutorials/distribute/custom_training) for more details.

name: Optional name for the op.

abstract compute(y_true: tensorflow.python.framework.ops.Tensor, y_pred: tensorflow.python.framework.ops.Tensor, **kwargs)[source]
Parameters

  • y_true –

  • y_pred –

  • kwargs –

Returns

normalize_weights(labels, weights)[source]
Parameters

  • labels –

  • weights –

Returns

static validate_inputs(y_true: tensorflow.python.framework.ops.Tensor, y_pred: tensorflow.python.framework.ops.Tensor)[source]

Make sure the inputs have rank 2 and are the same shape.

Parameters

  • y_true –

  • y_pred –

Returns

class deletor.losses.RankingCrossEntropy(normalize: bool = True, reduce: bool = True, lambda_weights: deletor.losses.LambdaWeight = None, pad_value=- 3.4028235e+38)[source]

Bases: deletor.losses.ListwiseLoss

Parameters

  • normalize –

  • reduce –

  • lambda_weights –

  • pad_value –

compute(y_true: tensorflow.python.framework.ops.Tensor, y_pred: tensorflow.python.framework.ops.Tensor, **kwargs)[source]
Parameters

  • y_true –

  • y_pred –

  • kwargs –

Returns

class deletor.losses.RankingSoftmax(reduce: bool = True, lambda_weights: deletor.losses.LambdaWeight = None, pad_value=- 3.4028235e+38, epsilon=1e-10)[source]

Bases: deletor.losses.ListwiseLoss

Ported from tensorflow_ranking. I assume this is the loss from Ai et al. (2019), which I thought could be implemented as RankingCrossEntropy.

Parameters

  • name –

  • lambda_weight –

  • params –

compute(y_true: tensorflow.python.framework.ops.Tensor, y_pred: tensorflow.python.framework.ops.Tensor, **kwargs)[source]
Parameters

  • y_true –

  • y_pred –

  • kwargs –

Returns

compute_unreduced(labels: tensorflow.python.framework.ops.Tensor, logits: tensorflow.python.framework.ops.Tensor)[source]
precompute(labels: tensorflow.python.framework.ops.Tensor, logits: tensorflow.python.framework.ops.Tensor)[source]
deletor.losses.apply_pairwise_op(op, tensor: tensorflow.python.framework.ops.Tensor)[source]
Parameters

  • op –

  • tensor –

Returns

deletor.losses.check_tensor_shapes(tensors: List[tensorflow.python.framework.ops.Tensor])[source]

Checks the tensor shapes to be compatible.

deletor.losses.get_valid_pairs_and_clean_labels(labels: tensorflow.python.framework.ops.Tensor)[source]

Returns a boolean Tensor for valid pairs and cleaned labels.

deletor.losses.masked_softmax(x: tensorflow.python.framework.ops.Tensor, mask: tensorflow.python.framework.ops.Tensor)[source]
Parameters

  • x –

  • mask –

Returns

deletor.metrics module

class deletor.metrics.AggregateMetric(metric: deletor.metrics.Metric)[source]

Bases: abc.ABC

abstract aggregate(y_true: tensorflow.python.framework.ops.Tensor, y_pred: tensorflow.python.framework.ops.Tensor)[source]
property name
class deletor.metrics.DiscountedCumulativeGain(k: Optional[int] = None, gain_fn: Callable = None, discount_fn: Callable = None, pad_value=- 3.4028235e+38)[source]

Bases: deletor.metrics.Metric

compute(y_true: tensorflow.python.framework.ops.Tensor, y_pred: tensorflow.python.framework.ops.Tensor, weights: tensorflow.python.framework.ops.Tensor = None, **kwargs)[source]
static default_discount(p)[source]
static default_gain(relevance)[source]
classmethod make_rank_indices(ranking: tensorflow.python.framework.ops.Tensor)[source]
property name
classmethod rank_by_scores(y_true: tensorflow.python.framework.ops.Tensor, y_pred: tensorflow.python.framework.ops.Tensor)[source]
class deletor.metrics.Mean(metric: deletor.metrics.Metric)[source]

Bases: deletor.metrics.AggregateMetric

aggregate(y_true: tensorflow.python.framework.ops.Tensor, y_pred: tensorflow.python.framework.ops.Tensor)[source]
class deletor.metrics.Metric[source]

Bases: abc.ABC

Base class for metric function classes. The compute method of implementing classes should return a metric value for each instance in a batch. Use an AggregateMetric to reduce the individual scores to a single number.

abstract compute(y_true: tensorflow.python.framework.ops.Tensor, y_pred: tensorflow.python.framework.ops.Tensor, weights: tensorflow.python.framework.ops.Tensor = None, **kwargs)[source]
property name
static validate_inputs(y_true: tensorflow.python.framework.ops.Tensor, y_pred: tensorflow.python.framework.ops.Tensor)[source]

Make sure the inputs have rank 2 and are the same shape.

Parameters

  • y_true –

  • y_pred –

Returns

class deletor.metrics.NormalizedDiscountedCumulativeGain(k: Optional[int] = None, gain_fn: Callable = None, discount_fn: Callable = None)[source]

Bases: deletor.metrics.Metric

compute(y_true: tensorflow.python.framework.ops.Tensor, y_pred: tensorflow.python.framework.ops.Tensor, weights: tensorflow.python.framework.ops.Tensor = None, **kwargs)[source]
property name
class deletor.metrics.Sum(metric: deletor.metrics.Metric)[source]

Bases: deletor.metrics.AggregateMetric

aggregate(y_true: tensorflow.python.framework.ops.Tensor, y_pred: tensorflow.python.framework.ops.Tensor)[source]

deletor.preprocessing module

Documentation

class deletor.preprocessing.DataTransformer[source]

Bases: abc.ABC

abstract inverse_transform(x: tensorflow.python.framework.ops.Tensor)[source]
abstract transform(x: tensorflow.python.framework.ops.Tensor)[source]
class deletor.preprocessing.MinMaxScaler(min_adjust: numpy.ndarray, scale: numpy.ndarray, data_min: numpy.ndarray, data_max: numpy.ndarray, data_range: numpy.ndarray, n_samples: int)[source]

Bases: deletor.preprocessing.DataTransformer

inverse_transform(x: tensorflow.python.framework.ops.Tensor)[source]
transform(x: tensorflow.python.framework.ops.Tensor)[source]
class deletor.preprocessing.PowerTransformer(lambdas: numpy.ndarray, scaler=None)[source]

Bases: deletor.preprocessing.DataTransformer

inverse_transform(x: tensorflow.python.framework.ops.Tensor)[source]
transform(x: tensorflow.python.framework.ops.Tensor)[source]
class deletor.preprocessing.RobustScaler(center: numpy.ndarray, scale: numpy.ndarray, with_centering: bool = True, with_scaling: bool = True)[source]

Bases: deletor.preprocessing.DataTransformer

inverse_transform(x: tensorflow.python.framework.ops.Tensor)[source]
transform(x: tensorflow.python.framework.ops.Tensor)[source]
class deletor.preprocessing.StandardScaler(scale: numpy.ndarray, mean: numpy.ndarray, var: numpy.ndarray, n_samples: int, with_mean: bool = True, with_std: bool = True)[source]

Bases: deletor.preprocessing.DataTransformer

inverse_transform(x: tensorflow.python.framework.ops.Tensor)[source]
transform(x: tensorflow.python.framework.ops.Tensor)[source]

deletor.tfutils module

Documentation

deletor.tfutils.disable_gpu()[source]
deletor.tfutils.grow_memory()[source]
deletor.tfutils.set_device(device: int)[source]
deletor.tfutils.to_nd_indices(indices)[source]
Parameters

indices – A Tensor of shape [batch_size, size] with integer values.

Returns

Module contents

Documentation