libEnsemble¶

libEnsemble is a complete Python toolkit for steering dynamic ensembles of calculations. Workflows are highly portable and detect/integrate heterogeneous resources with little effort. For instance, libEnsemble can automatically detect, assign, and reassign allocated processors and GPUs to ensemble members.

Users select or supply generator and simulator functions to express their ensembles; the generator typically steers the ensemble based on prior simulator results. Such functions can also launch and monitor external executables at any scale.

The libEnsemble docs has an online Perlmutter guide.

See this video for a demonstration workflow that coordinates GPU application instances on Perlmutter.

Code Sample¶

Run the following via python this_file.py --comms local --nworkers 4

import numpy as np

from libensemble import Ensemble
from libensemble.gen_funcs.sampling import uniform_random_sample
from libensemble.sim_funcs.six_hump_camel import six_hump_camel
from libensemble.specs import ExitCriteria, GenSpecs, SimSpecs
from libensemble.tools import add_unique_random_streams

if __name__ == "__main__":

    sampling = Ensemble(parse_args=True)
    sampling.sim_specs = SimSpecs(
        sim_f=six_hump_camel,
        inputs=["x"],
        outputs=[("f", float)],
    )

    sampling.gen_specs = GenSpecs(
        gen_f=uniform_random_sample,
        outputs=[("x", float, (2,))],
        user={
            "gen_batch_size": 500,
            "lb": np.array([-3, -2]),
            "ub": np.array([3, 2]),
        },
    )

    sampling.persis_info = add_unique_random_streams({}, sampling.nworkers + 1)
    sampling.exit_criteria = ExitCriteria(sim_max=101)
    sampling.run()
    sampling.save_output(__file__)

    if sampling.is_manager:
        print("Some output data:\n", sampling.H[["x", "f"]][:10])

Installing libEnsemble¶

Begin by loading the python module:

module load python

Create a conda virtual environment:

conda create -n my_environment python=3.10 -y

Activate your virtual environment:

export PYTHONNOUSERSITE=1
conda activate my_environment

Then either install via pip:

pip install libensemble

or via conda:

conda config --add channels conda-forge
conda install -c conda-forge libensemble

Other installation options are described in Advanced Installation.