Backends¶
Starting with version 3, emcee has an interface for serializing the sampler output. This can be useful in any scenario where you want to share the results of sampling or when sampling with an expensive model because, even if the sampler crashes, the current state of the chain will always be saved.
There is currently one backend that can be used to serialize the chain to a
file: emcee.backends.HDFBackend
.
The methods and options for this backend are documented below.
It can also be used as a reader for existing samplings.
For example, if a chain was saved using the backends.HDFBackend
, the
results can be accessed as follows:
reader = emcee.backends.HDFBackend("chain_filename.h5", read_only=True)
flatchain = reader.get_chain(flat=True)
The read_only
argument is not required, but it will make sure that you
don’t inadvertently overwrite the samples in the file.
-
class
emcee.backends.
Backend
(dtype=None)¶ A simple default backend that stores the chain in memory
-
get_autocorr_time
(discard=0, thin=1, **kwargs)¶ Compute an estimate of the autocorrelation time for each parameter
Parameters: - thin (Optional[int]) – Use only every
thin
steps from the chain. The returned estimate is multiplied bythin
so the estimated time is in units of steps, not thinned steps. (default:1
) - discard (Optional[int]) – Discard the first
discard
steps in the chain as burn-in. (default:0
)
Other arguments are passed directly to
emcee.autocorr.integrated_time()
.Returns: - The integrated autocorrelation time estimate for the
- chain for each parameter.
Return type: array[ndim] - thin (Optional[int]) – Use only every
-
get_blobs
(**kwargs)¶ Get the chain of blobs for each sample in the chain
Parameters: - flat (Optional[bool]) – Flatten the chain across the ensemble.
(default:
False
) - thin (Optional[int]) – Take only every
thin
steps from the chain. (default:1
) - discard (Optional[int]) – Discard the first
discard
steps in the chain as burn-in. (default:0
)
Returns: The chain of blobs.
Return type: array[.., nwalkers]
- flat (Optional[bool]) – Flatten the chain across the ensemble.
(default:
-
get_chain
(**kwargs)¶ Get the stored chain of MCMC samples
Parameters: - flat (Optional[bool]) – Flatten the chain across the ensemble.
(default:
False
) - thin (Optional[int]) – Take only every
thin
steps from the chain. (default:1
) - discard (Optional[int]) – Discard the first
discard
steps in the chain as burn-in. (default:0
)
Returns: The MCMC samples.
Return type: array[.., nwalkers, ndim]
- flat (Optional[bool]) – Flatten the chain across the ensemble.
(default:
-
get_last_sample
()¶ Access the most recent sample in the chain
-
get_log_prob
(**kwargs)¶ Get the chain of log probabilities evaluated at the MCMC samples
Parameters: - flat (Optional[bool]) – Flatten the chain across the ensemble.
(default:
False
) - thin (Optional[int]) – Take only every
thin
steps from the chain. (default:1
) - discard (Optional[int]) – Discard the first
discard
steps in the chain as burn-in. (default:0
)
Returns: The chain of log probabilities.
Return type: array[.., nwalkers]
- flat (Optional[bool]) – Flatten the chain across the ensemble.
(default:
-
grow
(ngrow, blobs)¶ Expand the storage space by some number of samples
Parameters: - ngrow (int) – The number of steps to grow the chain.
- blobs – The current list of blobs. This is used to compute the dtype for the blobs array.
-
has_blobs
()¶ Returns
True
if the model includes blobs
-
reset
(nwalkers, ndim)¶ Clear the state of the chain and empty the backend
Parameters: - nwakers (int) – The size of the ensemble
- ndim (int) – The number of dimensions
-
save_step
(state, accepted)¶ Save a step to the backend
Parameters: - state (State) – The
State
of the ensemble. - accepted (ndarray) – An array of boolean flags indicating whether or not the proposal for each walker was accepted.
- state (State) – The
-
shape
¶ The dimensions of the ensemble
(nwalkers, ndim)
-
-
class
emcee.backends.
HDFBackend
(filename, name='mcmc', read_only=False, dtype=None)¶ A backend that stores the chain in an HDF5 file using h5py
Note
You must install h5py to use this backend.
Parameters: - filename (str) – The name of the HDF5 file where the chain will be saved.
- name (str; optional) – The name of the group where the chain will be saved.
- read_only (bool; optional) – If
True
, the backend will throw aRuntimeError
if the file is opened with write access.
-
get_autocorr_time
(discard=0, thin=1, **kwargs)¶ Compute an estimate of the autocorrelation time for each parameter
Parameters: - thin (Optional[int]) – Use only every
thin
steps from the chain. The returned estimate is multiplied bythin
so the estimated time is in units of steps, not thinned steps. (default:1
) - discard (Optional[int]) – Discard the first
discard
steps in the chain as burn-in. (default:0
)
Other arguments are passed directly to
emcee.autocorr.integrated_time()
.Returns: - The integrated autocorrelation time estimate for the
- chain for each parameter.
Return type: array[ndim] - thin (Optional[int]) – Use only every
-
get_blobs
(**kwargs)¶ Get the chain of blobs for each sample in the chain
Parameters: - flat (Optional[bool]) – Flatten the chain across the ensemble.
(default:
False
) - thin (Optional[int]) – Take only every
thin
steps from the chain. (default:1
) - discard (Optional[int]) – Discard the first
discard
steps in the chain as burn-in. (default:0
)
Returns: The chain of blobs.
Return type: array[.., nwalkers]
- flat (Optional[bool]) – Flatten the chain across the ensemble.
(default:
-
get_chain
(**kwargs)¶ Get the stored chain of MCMC samples
Parameters: - flat (Optional[bool]) – Flatten the chain across the ensemble.
(default:
False
) - thin (Optional[int]) – Take only every
thin
steps from the chain. (default:1
) - discard (Optional[int]) – Discard the first
discard
steps in the chain as burn-in. (default:0
)
Returns: The MCMC samples.
Return type: array[.., nwalkers, ndim]
- flat (Optional[bool]) – Flatten the chain across the ensemble.
(default:
-
get_last_sample
()¶ Access the most recent sample in the chain
-
get_log_prob
(**kwargs)¶ Get the chain of log probabilities evaluated at the MCMC samples
Parameters: - flat (Optional[bool]) – Flatten the chain across the ensemble.
(default:
False
) - thin (Optional[int]) – Take only every
thin
steps from the chain. (default:1
) - discard (Optional[int]) – Discard the first
discard
steps in the chain as burn-in. (default:0
)
Returns: The chain of log probabilities.
Return type: array[.., nwalkers]
- flat (Optional[bool]) – Flatten the chain across the ensemble.
(default:
-
grow
(ngrow, blobs)¶ Expand the storage space by some number of samples
Parameters: - ngrow (int) – The number of steps to grow the chain.
- blobs – The current list of blobs. This is used to compute the dtype for the blobs array.
-
has_blobs
()¶ Returns
True
if the model includes blobs
-
reset
(nwalkers, ndim)¶ Clear the state of the chain and empty the backend
Parameters: - nwakers (int) – The size of the ensemble
- ndim (int) – The number of dimensions
-
save_step
(state, accepted)¶ Save a step to the backend
Parameters: - state (State) – The
State
of the ensemble. - accepted (ndarray) – An array of boolean flags indicating whether or not the proposal for each walker was accepted.
- state (State) – The
-
shape
¶ The dimensions of the ensemble
(nwalkers, ndim)