nums.core.storage.utils module

class nums.core.storage.utils.Batch(total_size, batch_size)[source]

Bases: object

Simple object for creating an object that can generate batches of sequential integers.

classmethod from_num_batches(total_size, num_batches)[source]
get_batches(total_size, batch_size)[source]
Parameters

  • total_size – Total number of items to split into batches.

  • batch_size – Size of each batch.

Returns

A list of 2-tuples. Each 2-tuple is a segment of indices corresponding to items of size batch_size. The size of the list is total_size / batch_size.

nums.core.storage.utils.get_char_encoding(fname)[source]
nums.core.storage.utils.get_file_size(fname)[source]
nums.core.storage.utils.get_np_comments(fname, comments)[source]
nums.core.storage.utils.get_np_txt_info(fname, comments, delimiter)[source]
nums.core.storage.utils.reverse_readline(filename, buf_size=8192)[source]

A generator that returns the lines of a file in reverse order