Documentation of the supplied Libraries

StreamingBandit is build on a philosophy of streaming (or online, or row-by-row) updating of parameters of a policy. The libs libraries provide some functionality for estimating and fitting statistical models online. These libraries are available directly in the getaction and setreward code.

The following code provides an example of the use of one of the libs in the getaction code of an experiment:

1
2
3
4
5
 mean_list = base.List(
              self.get_theta(key="treatment"),
              base.Mean, ["control", "treatment"]
              )
 self.action["treatment"] = meanList.max()

which would retrieve a list of sample means by name (treatment and control), and subsequently select the name with the highest mean.

Note that each of the libs provides update methods to update the object when new data is observed. For example, the code:

1
2
3
4
5
 mean = base.Mean(
      self.get_theta(
      key="treatment", value=self.action["treatment"])
      )
 mean.update(self.reward["value"])

can be used in the setreward code to retrieve the mean specified in the treatment variable, and subsequently update it using the value.

Below we detail the current libs and each of their methods.

Base classes

The base classes provide the functionality to create, store, and update (lists of) counts, means, variances, proportions, and co-variances in data streams.

class libs.base.Correlation(default)

Class to represent a correlation using an online estimator.

Variables

default (dict) – A dictionary that consists of a counter n, mean for x x_bar, mean for y y_bar, standard deviation for x x_s, standard deviation for y y_s, variance for x x_v, variance for y y_v, covariance cov and correlation c. Leave empty to start a new correlation.

__add__(other)

Override the arithmetic + function.

__init__(default)

Construct a new strmBase object.

__mul__(other)

Override the arithmetic * function.

__sub__(other)

Override the arithmetic - function.

__truediv__(other)

Override the arithmetic / function.

update(value)

Adds value to correlation.

Parameters

value (dict) – A dict of ints of x and y.

class libs.base.Count(default)

Class to represent a counter using an online estimator.

Variables

default (dict) – A dictionary that consists of the counter n. Leave empty to start a new counter.

__init__(default)

Construct a new strmBase object.

increment()

Update the counter with value 1.

update(value=1)

Adds value to the counter.

class libs.base.Covariance(default)

Class to represent a covariance using an online estimator.

Variables

default (dict) – A dictionary that consists of a counter n, mean for x x_bar, mean for y y_bar and covariance cov. Leave empty to start a new covariance.

__add__(other)

Override the arithmetic + function.

__init__(default)

Construct a new strmBase object.

__mul__(other)

Override the arithmetic * function.

__sub__(other)

Override the arithmetic - function.

__truediv__(other)

Override the arithmetic / function.

update(value)

Adds value to covariance.

Parameters

value (dict) – A dict of ints of x and y.

class libs.base.List(objects, _t, value_names)

Class to represent a list of Base classes. Here you can store multiple Base classes to simplify your AB test and whatnot.

Variables

  • objects (dict) – A dict of dicts with the objects in dictionary form - so not a class instance!

  • _t (type) – The type of Base class (e.g. Proportion, Count).

  • value_names (list) – The value names to be grabbed from the objects dict. This is used for e.g. random picks.

__init__(objects, _t, value_names)

Initialize self. See help(type(self)) for accurate signature.

__weakref__

list of weak references to the object (if defined)

count()

Checks if the Base class has a counter. If that’s the case, it will add all the counters and return the total count.

get_dict()

Returns each Base class in the objects as a dict in a dict.

get_value()

Returns the main value of each Base class in the objects in a dict.

max()

Finds the maximum main value from all Base classes. If no max is available yet (because all main values are 0), it will return a random max.

min()

Finds the minimum main value from all Base classes. If there are multiple minima available (for example because all main values are 0) it will return a random min

random()

Return a random choice from the value_names list.

class libs.base.Mean(default)

Class to represent a mean using an online estimator.

Variables

default (dict) – A dictionary that consists of a counter n and mean m. Leave empty to start a new mean.

__init__(default)

Construct a new strmBase object.

get_count()

Returns the counter of the mean object.

update(value)

Adds value to the mean.

Parameters

value (int) – The value of x to update the mean.

class libs.base.Proportion(default)

Class to represent a proportion using an online estimator.

Variables

default (dict) – A dictionary that consists of a counter n, and proportion p. Leave empty to start a new proportion.

__add__(other)

Override the arithmetic + function.

__init__(default)

Construct a new strmBase object.

__mul__(other)

Override the arithmetic * function.

__sub__(other)

Override the arithmetic - function.

__truediv__(other)

Override the arithmetic / function.

update(value)

Adds value to the proportion.

Parameters

value (int) – A value 0 or 1.

class libs.base.Variance(default)

Class to represent a variance using an online estimator.

Variables

default (dict) – A dictionary that consists of a counter n, mean x_bar, standard deviation s and variance v. Leave empty to start a new variance.

__add__(other)

Override the arithmetic + function.

__init__(default)

Construct a new strmBase object.

__mul__(other)

Override the arithmetic * function.

__sub__(other)

Override the arithmetic - function.

__truediv__(other)

Override the arithmetic / function.

update(value)

Adds value to the variance.

Parameters

value (int) – The value used for updating.

Linear Regression

The linear regression library allows online fitting of a linear regression model using OLS.

class libs.lm.LM(default, p=None, add_intercept=False)

Class to interpet a linear model.

Variables

  • default (dict) – The value of the model, consisting of a 1*p list of b, p*p list of lists A and counter n.

  • p (int) – Number of estimators.

  • add_intercept (bool) – If True, the update function expects that there it has to add the intercept to the X vector itself.

__init__(default, p=None, add_intercept=False)

Class to fit a linear model with a sequential solution for linear regression.

__weakref__

list of weak references to the object (if defined)

get_coefs()

Returns the coefficients beta as a numpy array.

get_dict()

Return all the variables that are needed to do an online estimation in a dictionary.

predict(x)

Predict the output/observation value based on values for the regressors.

Parameters

  • x (list) – A list of ints of the regressors.

  • y (numpy) – A numpy array of the predicted observation.

update(y, x, discount=1)

Update the linear model.

Parameters

  • y (int) – The observation value.

  • x (list) – A list of ints of the regressors.

Thompson sampling

This library implements Thompson sampling for the k-armed Bernoulli bandit as well as Thompson sampling for optimal design of experiments (Thompson sampling based on the posterior variance of the observed outcome). See also this paper.

class libs.thompson.BBThompsonList(objects, value_names)

Class to draw decisions using a Bernoulli Bandit Thompson sampler.

Variables

  • objects (dict) – A dict of dict of thetas (which should be proportions, see documentation of Proportions on how it should look like.)

  • value_names (list) – A list with the possible value names for the actions

__init__(objects, value_names)

Create an instance of a BB Thompson Sampler.

propensity(value, n=1000)

Calculate propensity of given arm using n draws.

Returns int propensity

The propensity of the arm.

thompson()

Draw decision using the Bernoulli Bandit Thompson sampler.

Returns string choice

The choice of action that’s made.

class libs.thompson.ThompsonVarList(objects, value_names)

Class to allocate observations in 2 group a within subjects desing such as to maximize the precision of the estimated effect size (assuming unequal variance of the two groups)

__init__(objects, value_names)

Create instance of a Thompson Variance Sampler by creating a variance list

experimentThompson()

Obtain draw and allocate to minimize estimation precision

Bootstrapped Thompson Sampling

This library provides the functionality to perform on online bootstrap of any streaming estimate obtained by the libs and subsequently implements Bootstrap Thompson Sampling (BTS, see this paper for details).

class libs.bts.BTS(params, update_method=None, m=100, default_params=None, param_noise=True, noisesd=0.1)

Class to implement BTS.

Variables

  • params (dict) – If initialized, a dict of dicts, containing m samples of parameters for the update_method/model that is being used. If not initialized this will be made into a dict of dicts using the given params or default_params.

  • update_method (Class) – A class reference containing the update method. This will be checked and if it is a class, it will be used to update the parameters.

  • m (int) – The number of bootstraps that BTS uses.

  • default_params (dict) – A dictionary containing the default parameters. If the params variable is empty/not long enough, this will be used.

  • param_noise (bool) – If True, BTS will add parameter noise to all the lists in the default_params dictionary when initializing the bootstraps.

  • noisesd (double) – If param_noise is used, this is the standard deviation used for adding noise using a Normal distribution.

__init__(params, update_method=None, m=100, default_params=None, param_noise=True, noisesd=0.1)

Initialize self. See help(type(self)) for accurate signature.

__weakref__

list of weak references to the object (if defined)

get_dict()

Return all the variables and bootstraps that are needed to do an online estimation in a dictionary. Or to save the data into a database.

sample()

Return a sample of the m bootstraps.

update(y, x, *args, **kwargs)

Update the bootstraps using a double or nothing update policy.

Parameters

  • y – This may contain any type of class as long as the update_method provided is able to parse this type.

  • x – This may contain any type of class as long as the update_method provided is able to parse this type.

Lock-in Feedback

This library provides a (1D) implementation of Lock-in Feedback. See this paper for details.

class libs.lif.LiF(theta, x0=1.0, a=1.4, t=100, gamma=0.004, omega=0.8, lifversion=2)

Class implementing the Lock in Feedback algorithm. Implementation of Lock in Feedback, following the definition by Maurits Kaptein and Davide Iannuzzi in https://arxiv.org/abs/1502.00598.

Variables

  • theta (dict) – Theta, consisting of Yw, t, and x0.

  • x0 (float) – Start value of X.

  • A (float) – Amplitude.

  • T (int) – Integration time.

  • gamma (float) – Learnrate.

  • omega (float) – Omega.

  • lifversion (int) – Either version 1 or 2.

__init__(theta, x0=1.0, a=1.4, t=100, gamma=0.004, omega=0.8, lifversion=2)

Class implementing the Lock in Feedback algorithm.

__weakref__

list of weak references to the object (if defined)

get_dict()

Return dict theta.

Returns

Theta, consisting of Yw, t, and x0.

suggest()

Returns dict containing controlled variable x oscillating with time t, the value of t itself and the current x0.

Returns

Suggestion {x,t,x0}

update(t, x, y, x0)

Update LiF with outcome y at time t and at value x integrating yω over a time T = 2πN.

Parameters

  • t (int) – time t.

  • x (float) – controlled variable x.

  • y (float) – outcome variable y.

Returns

True

Thompson Bayesian Linear Regression

This library implements (online) Bayesian linear regression and Thompson sampling based on the posterior Beta’s.

class libs.thompson_bayesian_linear.ThompsonBayesianLinear(default)

Class for Thompson sampling for Bayesian Linear Regression

Variables

default (dict) – The value of the model, consisting of a 1*p list of J, p*p list of P and an error rate.

__init__(default)

Initialize self. See help(type(self)) for accurate signature.

__weakref__

list of weak references to the object (if defined)

get_dict()

Return all the variables that are needed to do an online estimation in a dictionary. Or to save the parameters in a database.

sample()

Return a sample of coefficients Betas using Thompson sampling.

update(y, x, discount=1)

Update the Bayesian linear model.

Parameters

  • y (int) – The observation value.

  • x (list) – A list of ints of the regressors.

  • discount (int) – A discount. Default is 1, which means no discount is used.