Skip to content

Optimizers

bocoel.core.optim

Optimizers is much like optimizers in PyTorch, but for the purpose of optimizing queries and search. Each optimizer would perform a few steps that collectively would guide the search towards the optimal trajectory.

bocoel.Optimizer

Optimizer(index_eval: IndexEvaluator, index: Index, **kwargs: Any)

Bases: Protocol

The protocol for optimizers. Optimizers are used for optimizing the search space, Find the best exploration sequence for a given task.

Parameters:

Name Type Description Default
index_eval IndexEvaluator

The id evaluator. Evalutes the items at the given storage indices.

required
index Index

The index that contains information about the domain.

required
**kwargs Any

The keyword arguments.

{}
Source code in src/bocoel/core/optim/interfaces/optim.py
19
20
21
22
23
24
25
26
27
28
def __init__(self, index_eval: IndexEvaluator, index: Index, **kwargs: Any) -> None:
    """
    Parameters:
        index_eval: The id evaluator. Evalutes the items at the given storage indices.
        index: The index that contains information about the domain.
        **kwargs: The keyword arguments.
    """

    # Included s.t. constructors of Index can be used.
    ...

task abstractmethod property

task: Task

The task to use for the optimization.

Returns:

Type Description
Task

One of Task.EXPLORE or Task.MINIMIZE or Task.MAXIMIZE.

step abstractmethod

step() -> Mapping[int, float]

Perform a single step of optimization. This is a shortcut into the optimization process. For methods that evaluate the entire search at once, this method would output the slices of the entire search.

Returns:

Type Description
Mapping[int, float]

A mapping of storage indices to the corresponding scores.

Raises:

Type Description
StopIteration

If the optimization is complete.

Source code in src/bocoel/core/optim/interfaces/optim.py
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
@abc.abstractmethod
def step(self) -> Mapping[int, float]:
    """
    Perform a single step of optimization.
    This is a shortcut into the optimization process.
    For methods that evaluate the entire search at once,
    this method would output the slices of the entire search.

    Returns:
        A mapping of storage indices to the corresponding scores.

    Raises:
        StopIteration: If the optimization is complete.
    """

    ...

bocoel.AxServiceOptimizer

AxServiceOptimizer(
    index_eval: IndexEvaluator,
    index: Index,
    *,
    sobol_steps: int = 0,
    device: Device = "cpu",
    workers: int = 1,
    task: Task = Task.EXPLORE,
    acqf: str | AcquisitionFunc = AcquisitionFunc.AUTO,
    surrogate: str | SurrogateModel = SurrogateModel.AUTO,
    surrogate_kwargs: SurrogateOptions | None = None
)

Bases: Optimizer

The Ax optimizer that uses the service API. See https://ax.dev/tutorials/gpei_hartmann_service.html

Parameters:

Name Type Description Default
index_eval IndexEvaluator

The evaluator to use for the query.

required
index Index

The index to for querying.

required
sobol_steps int

The number of steps to use for the Sobol sequence.

0
device Device

The device to use for the optimization.

'cpu'
workers int

The number of workers to use for the optimization.

1
task Task

The task to use for the optimization.

EXPLORE
acqf str | AcquisitionFunc

The acquisition function to use for the optimization.

AUTO
surrogate str | SurrogateModel

The surrogate model to use for the optimization.

AUTO
surrogate_kwargs SurrogateOptions | None

The keyword arguments to pass to the surrogate model.

None
Source code in src/bocoel/core/optim/ax/optim.py
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
def __init__(
    self,
    index_eval: IndexEvaluator,
    index: Index,
    *,
    sobol_steps: int = 0,
    device: Device = "cpu",
    workers: int = 1,
    task: Task = Task.EXPLORE,
    acqf: str | AcquisitionFunc = AcquisitionFunc.AUTO,
    surrogate: str | SurrogateModel = SurrogateModel.AUTO,
    surrogate_kwargs: SurrogateOptions | None = None,
) -> None:
    """
    Parameters:
        index_eval: The evaluator to use for the query.
        index: The index to for querying.
        sobol_steps: The number of steps to use for the Sobol sequence.
        device: The device to use for the optimization.
        workers: The number of workers to use for the optimization.
        task: The task to use for the optimization.
        acqf: The acquisition function to use for the optimization.
        surrogate: The surrogate model to use for the optimization.
        surrogate_kwargs: The keyword arguments to pass to the surrogate model.
    """

    silence_ax()

    acqf = AcquisitionFunc.lookup(acqf)
    task = Task.lookup(task)

    utils.check_acquisition_task_combo(acqf=acqf, task=task)

    self._device = device
    self._acqf = acqf
    self._surrogate = SurrogateModel.lookup(surrogate).surrogate(surrogate_kwargs)
    self._task = task

    self._ax_client = AxClient(generation_strategy=self._gen_strat(sobol_steps))
    self._create_experiment(index.boundary)

    self._index_eval = index_eval
    self._index = index
    self._workers = workers
    self._terminate = False

bocoel.Task

Bases: StrEnum

EXPLORE class-attribute instance-attribute

EXPLORE = 'EXPLORE'

MINIMIZE class-attribute instance-attribute

MINIMIZE = 'MINIMIZE'

MAXIMIZE class-attribute instance-attribute

MAXIMIZE = 'MAXIMIZE'

lookup classmethod

lookup(name: str | Self) -> Self
Source code in src/bocoel/common/enums.py
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
@classmethod
def lookup(cls, name: str | Self) -> Self:
    if isinstance(name, cls):
        return name

    try:
        return cls[name]
    except KeyError:
        pass

    try:
        return cls(name)
    except ValueError:
        pass

    raise ItemNotFound(f"Item not found in enum. Must be one of {list(cls)}")

bocoel.AcquisitionFunc

Bases: StrEnum

ENTROPY class-attribute instance-attribute

ENTROPY = 'ENTROPY'

MES class-attribute instance-attribute

MES = 'MAX_VALUE_ENTROPY'

UCB class-attribute instance-attribute

UCB = 'UPPER_CONFIDENCE_BOUND'

QUCB class-attribute instance-attribute

QUCB = 'QUASI_UPPER_CONFIDENCE_BOUND'

EI class-attribute instance-attribute

EI = 'EXPECTED_IMPROVEMENT'

QEI class-attribute instance-attribute

QEI = 'QUASI_EXPECTED_IMPROVEMENT'

AUTO class-attribute instance-attribute

AUTO = 'AUTO'

botorch_acqf_class property

botorch_acqf_class: type[AcquisitionFunction] | None

lookup classmethod

lookup(name: str | Self) -> Self
Source code in src/bocoel/common/enums.py
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
@classmethod
def lookup(cls, name: str | Self) -> Self:
    if isinstance(name, cls):
        return name

    try:
        return cls[name]
    except KeyError:
        pass

    try:
        return cls(name)
    except ValueError:
        pass

    raise ItemNotFound(f"Item not found in enum. Must be one of {list(cls)}")

bocoel.core.optim.ax.surrogates.SurrogateModel

Bases: StrEnum

SAAS class-attribute instance-attribute

SAAS = 'SAAS'

AUTO class-attribute instance-attribute

AUTO = 'AUTO'

lookup classmethod

lookup(name: str | Self) -> Self
Source code in src/bocoel/common/enums.py
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
@classmethod
def lookup(cls, name: str | Self) -> Self:
    if isinstance(name, cls):
        return name

    try:
        return cls[name]
    except KeyError:
        pass

    try:
        return cls(name)
    except ValueError:
        pass

    raise ItemNotFound(f"Item not found in enum. Must be one of {list(cls)}")

surrogate

surrogate(surrogate_options: SurrogateOptions | None) -> Surrogate | None
Source code in src/bocoel/core/optim/ax/surrogates/supported.py
25
26
27
28
29
30
31
32
33
34
35
36
def surrogate(self, surrogate_options: SurrogateOptions | None) -> Surrogate | None:
    if surrogate_options is None:
        surrogate_options = {}

    match self:
        case SurrogateModel.AUTO:
            return None
        case SurrogateModel.SAAS:
            return Surrogate(
                botorch_model_class=SaasFullyBayesianSingleTaskGP,
                **surrogate_options,
            )

bocoel.core.optim.ax.surrogates.SurrogateOptions

Bases: TypedDict

mll_class instance-attribute

mll_class: NotRequired[type[MarginalLogLikelihood]]

mll_options instance-attribute

mll_options: NotRequired[MLLOptions]

bocoel.KMeansOptimizer

KMeansOptimizer(
    index_eval: IndexEvaluator,
    index: Index,
    *,
    batch_size: int,
    embeddings: NDArray,
    model_kwargs: KMeansOptions
)

Bases: ScikitLearnOptimizer

The KMeans optimizer that uses clustering algorithms.

Parameters:

Name Type Description Default
index_eval IndexEvaluator

The evaluator to use on the storage.

required
index Index

The index to use for the query.

required
batch_size int

The number of embeddings to evaluate at once.

required
embeddings NDArray

The embeddings to cluster.

required
model_kwargs KMeansOptions

The keyword arguments to pass to the KMeans model.

required
Source code in src/bocoel/core/optim/sklearn/kmeans.py
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
def __init__(
    self,
    index_eval: IndexEvaluator,
    index: Index,
    *,
    batch_size: int,
    embeddings: NDArray,
    model_kwargs: KMeansOptions,
) -> None:
    """
    Parameters:
        index_eval: The evaluator to use on the storage.
        index: The index to use for the query.
        batch_size: The number of embeddings to evaluate at once.
        embeddings: The embeddings to cluster.
        model_kwargs: The keyword arguments to pass to the KMeans model.
    """

    model = KMeans(**model_kwargs)

    super().__init__(
        index_eval=index_eval,
        index=index,
        embeddings=embeddings,
        model=model,
        batch_size=batch_size,
    )

    self._model_kwargs = model_kwargs

bocoel.KMeansOptions

Bases: TypedDict

n_clusters instance-attribute

n_clusters: int

init instance-attribute

init: NotRequired[Literal['k-means++', 'random']]

n_init instance-attribute

n_init: NotRequired[int | Literal['auto']]

tol instance-attribute

tol: NotRequired[float]

verbose instance-attribute

verbose: NotRequired[int]

random_state instance-attribute

random_state: NotRequired[int]

algorithm instance-attribute

algorithm: NotRequired[Literal['llyod', 'elkan']]

bocoel.KMedoidsOptimizer

KMedoidsOptimizer(
    index_eval: IndexEvaluator,
    index: Index,
    *,
    batch_size: int,
    embeddings: NDArray,
    model_kwargs: KMedoidsOptions
)

Bases: ScikitLearnOptimizer

The KMedoids optimizer that uses clustering algorithms.

Parameters:

Name Type Description Default
index_eval IndexEvaluator

The evaluator to use for the index.

required
index Index

The index to use for the query.

required
batch_size int

The number of embeddings to evaluate at once.

required
embeddings NDArray

The embeddings to cluster.

required
model_kwargs KMedoidsOptions

The keyword arguments to pass to the KMedoids model.

required
Source code in src/bocoel/core/optim/sklearn/kmedoids.py
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
def __init__(
    self,
    index_eval: IndexEvaluator,
    index: Index,
    *,
    batch_size: int,
    embeddings: NDArray,
    model_kwargs: KMedoidsOptions,
) -> None:
    """
    Parameters:
        index_eval: The evaluator to use for the index.
        index: The index to use for the query.
        batch_size: The number of embeddings to evaluate at once.
        embeddings: The embeddings to cluster.
        model_kwargs: The keyword arguments to pass to the KMedoids model.
    """

    # Optional dependency.
    from sklearn_extra.cluster import KMedoids

    model = KMedoids(**model_kwargs)

    super().__init__(
        index_eval=index_eval,
        index=index,
        embeddings=embeddings,
        model=model,
        batch_size=batch_size,
    )

    self._model_kwargs = model_kwargs

bocoel.KMedoidsOptions

Bases: TypedDict

n_clusters instance-attribute

n_clusters: int

metrics instance-attribute

metrics: NotRequired[str]

method instance-attribute

method: NotRequired[Literal['alternate', 'pam']]

init instance-attribute

init: NotRequired[Literal['random', 'heuristic', 'kmedoids++', 'build']]

max_iter instance-attribute

max_iter: NotRequired[int]

random_state instance-attribute

random_state: NotRequired[int]

bocoel.RandomOptimizer

RandomOptimizer(
    index_eval: IndexEvaluator, index: Index, *, samples: int, batch_size: int
)

Bases: Optimizer

The random optimizer that uses random search.

Parameters:

Name Type Description Default
index_eval IndexEvaluator

The evaluator to use for the storage.

required
index Index

The index to use for the query.

required
samples int

The number of samples to use for the optimization.

required
batch_size int

The number of samples to evaluate at once.

required
Source code in src/bocoel/core/optim/random.py
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
def __init__(
    self,
    index_eval: IndexEvaluator,
    index: Index,
    *,
    samples: int,
    batch_size: int,
) -> None:
    """
    Parameters:
        index_eval: The evaluator to use for the storage.
        index: The index to use for the query.
        samples: The number of samples to use for the optimization.
        batch_size: The number of samples to evaluate at once.
    """

    LOGGER.info("Instantiating RandomOptimizer", total=len(index), samples=samples)

    self._index_eval = index_eval
    self._index = index
    self._generator = iter(BatchedGenerator(self._gen_random(samples), batch_size))

bocoel.UniformOptimizer

UniformOptimizer(
    index_eval: IndexEvaluator,
    index: Index,
    *,
    grids: Sequence[int],
    batch_size: int
)

Bases: Optimizer

The uniform optimizer that uses grid-based search.

Parameters:

Name Type Description Default
index_eval IndexEvaluator

The evaluator to use for the storage.

required
index Index

The index to use for the query.

required
grids Sequence[int]

The number of grids to use for the optimization.

required
batch_size int

The number of grids to evaluate at once.

required
Source code in src/bocoel/core/optim/uniform.py
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
def __init__(
    self,
    index_eval: IndexEvaluator,
    index: Index,
    *,
    grids: Sequence[int],
    batch_size: int,
) -> None:
    """
    Parameters:
        index_eval: The evaluator to use for the storage.
        index: The index to use for the query.
        grids: The number of grids to use for the optimization.
        batch_size: The number of grids to evaluate at once.
    """

    LOGGER.info("Instantiating UnfiromOptimizer", grids=grids)

    self._index_eval = index_eval
    self._index = index

    self._generator = iter(BatchedGenerator(self._gen_locs(grids), batch_size))

    if len(grids) != self._index.dims:
        raise ValueError(f"Expected {self._index.dims} strides, got {grids}")

bocoel.IndexEvaluator

Bases: Protocol

A protocol for evaluating with the indices.

__call__ abstractmethod

__call__(idx: ArrayLike) -> NDArray

Evaluates the given batched query. The order of the results must be kept in the original order.

Parameters:

Name Type Description Default
idx ArrayLike

The indices to evaluate. Must be a 1D array.

required

Returns:

Type Description
NDArray

The indices of the results. Must be in the same order as the query.

Source code in src/bocoel/core/optim/interfaces/evals.py
62
63
64
65
66
67
68
69
70
71
72
73
74
75
@abc.abstractmethod
def __call__(self, idx: ArrayLike, /) -> NDArray:
    """
    Evaluates the given batched query.
    The order of the results must be kept in the original order.

    Parameters:
        idx: The indices to evaluate. Must be a 1D array.

    Returns:
        The indices of the results. Must be in the same order as the query.
    """

    ...

bocoel.QueryEvaluator

Bases: Protocol

A protocol for evaluating the query results.

__call__ abstractmethod

__call__(query: ArrayLike) -> OrderedDict[int, float]

Evaluates the given batched query. The order of the results must be kept in the original order.

Parameters:

Name Type Description Default
query ArrayLike

The query to evaluate.

required

Returns:

Type Description
OrderedDict[int, float]

The results of the query. Must be in the same order as the query.

Source code in src/bocoel/core/optim/interfaces/evals.py
41
42
43
44
45
46
47
48
49
50
51
52
53
54
@abc.abstractmethod
def __call__(self, query: ArrayLike, /) -> OrderedDict[int, float]:
    """
    Evaluates the given batched query.
    The order of the results must be kept in the original order.

    Parameters:
        query: The query to evaluate.

    Returns:
        The results of the query. Must be in the same order as the query.
    """

    ...

bocoel.SearchEvaluator

Bases: Protocol

A protocol for evaluating the search results.

__call__ abstractmethod

__call__(results: Mapping[int, SearchResult]) -> Mapping[int, float]

Evaluates the given batched search result. The order of the results must be kept in the original order.

Parameters:

Name Type Description Default
results Mapping[int, SearchResult]

The results of the search. Mapping from index to search result.

required

Returns:

Type Description
Mapping[int, float]

The results of the search. Must be in the same order as the query.

Source code in src/bocoel/core/optim/interfaces/evals.py
20
21
22
23
24
25
26
27
28
29
30
31
32
33
@abc.abstractmethod
def __call__(self, results: Mapping[int, SearchResult], /) -> Mapping[int, float]:
    """
    Evaluates the given batched search result.
    The order of the results must be kept in the original order.

    Parameters:
        results: The results of the search. Mapping from index to search result.

    Returns:
        The results of the search. Must be in the same order as the query.
    """

    ...