examples package

Subpackages

• examples.attn package
  • Submodules
  • examples.attn.mltr30k module
  • Module contents
• examples.gsf package
  • Submodules
  • examples.gsf.mltr30k module
  • examples.gsf.mltr30k_boost module
  • Module contents
• examples.mlp package
  • Submodules
  • examples.mlp.mltr30k module
  • Module contents

Submodules

examples.build_tfrecords module

A script to convert the MLTR data files in libsvm format into tfrecords.

examples.build_tfrecords.bytes_feature(value)

Returns a bytes_list from a string / byte.

examples.build_tfrecords.float_feature(value)

Returns a float_list from a float / double.

examples.build_tfrecords.int64_feature(value)

Returns an int64_list from a bool / enum / int / uint.

examples.build_tfrecords.make_command_line_options()

examples.build_tfrecords.make_dataset(args: argparse.Namespace)

examples.build_tfrecords.make_example(sequence)

examples.build_tfrecords.svm_generator(fh, current_sequence)

examples.build_tfrecords.write_data(args: argparse.Namespace)

examples.build_tfrecords.write_scalers(args: argparse.Namespace)

examples.pipeline module

Documentation

examples.pipeline.apply_map(fn, *datasets, **kwargs)

examples.pipeline.expand_dims_for_unbatch(x, y)

examples.pipeline.is_valid_query(_, y)

examples.pipeline.load_dataset(dataset_filename: str, scalers: List[str] = None, n_features: int = 136)

Load a tensorflow records dataset file into tf.Dataset object and optionally apply a scaling method to the dense features.

Parameters

• dataset_filename – The path where the dataset file is located.

• scalers – A two item list containing the information necessary to scale the data. The first element is the path to the scalers shelve file. The second element is the name of the scaler to use. It should be one of [minmax, standard, robust, power].

• n_features – The number of features in the dataset.

Returns

A tensorflow Dataset object.

examples.pipeline.make_feature_description(n_features: int)

Specify the feature schemas needed to parse the records.

Parameters

n_features – The number of sequential features in the dataset.

Returns

A tuple of dictionaries. The first item of the tuple is a dictionary containing the context features and the second item contains the sequence features. In this example there are no context features and all the sequence features are dense.

examples.pipeline.make_padded_shapes(list_size: Optional[int] = None, n_features: int = 136, sample_pre_batch: bool = False, with_weights: bool = False)

examples.pipeline.make_padding_values(sample_pre_batch: bool = False, with_weights: bool = False)

examples.pipeline.make_scaler(name: str, s)

examples.pipeline.pad_groups(x, y, group_size: int)

examples.pipeline.parse_example(proto, context_desc, sequence_desc)

Parses a single sequence example using the feature descriptions provided.

Parameters

• proto – The raw record from the tfrecords file.

• context_desc – The schema for the context features.

• sequence_desc – The schema for the sequential features.

Returns

examples.pipeline.select_features(x, y, indices)

Select a subset of the (sequential) features and overwrite the sequence_dense field of the input (x) dictionary.

Parameters

• x – The input data containing a dictionary of tensors.

• y – The target values.

• indices – The indices of the features to select.

Returns

examples.pipeline.shuffle_documents(x, y, seed: Optional[int] = None)

examples.pipeline.squeeze_for_unbatch(x, y)

examples.pipeline.truncate_document_list(x, y, list_size: int)

examples.utils module

Documentation

examples.utils.evaluate(net: tensorflow.python.keras.engine.training.Model, eval_data: tensorflow.python.data.ops.dataset_ops.DatasetV2, train_meta: Dict[str, Any])

examples.utils.log_level_type(s: str)

examples.utils.make_loss(args: argparse.Namespace, reduce: bool = True, **kwargs)

examples.utils.make_optimizer(args: argparse.Namespace)

examples.utils.train(model: tensorflow.python.keras.engine.training.Model, train_data: tensorflow.python.data.ops.dataset_ops.DatasetV2, train_meta: Dict[str, Any])

Train a model with the given training data.

Parameters

• model – The model to train.

• train_data – The training data.

• train_meta – A dictionary containing variables to store the results in.

Module contents

Documentation