History

The R6 class History keeps a log of all Simulator interactions in its internal data.table. It also provides basic data summaries, and can save or load simulation log data files.

Usage

History <- History$new(n = 1, save_context = FALSE, save_theta = FALSE)

Arguments

n

integer. The number of rows, to be preallocated during initialization.

save_context

logical. Save context matrix X when writing simulation data?

save_theta

logical. Save parameter lists theta when writing simulation data?

Methods

reset()

Resets a History instance to its original initialisation values.

insert(index, t, action, reward, agent_name, simulation_index, context_value = NA, theta_value = NA)

Saves one row of simulation data. Is generally not called directly, but from a Simulator instance.

save(filename = NA)

Writes the History log file in its default data.table format, with filename as the name of the file which the data is to be written to.

load = function(filename, interval = 0)

Reads a History log file in its default data.table format, with filename as the name of the file which the data are to be read from. If interval is larger than 0, every interval of data is read instead of the full data file. This can be of use with (a first) analysis of very large data files.

get_data_frame()

Returns the History log as a data.frame.

set_data_frame(df, auto_stats = TRUE)

Sets the History log with the data in data.frame dt. Recalculates cumulative statistics when auto_stats is TRUE.

get_data_table()

Returns the History log as a data.table.

set_data_table(dt, auto_stats = TRUE)

Sets the History log with the data in data.table dt. Recalculates cumulative statistics when auto_stats is TRUE.

clear_data_table()

Clear History's internal data.table log.

save_csv(filename = NA)

Saves History data to csv file.

extract_theta(limit_agents, parameter, arm, tail = NULL)

Extract theta parameter from theta list for limit_agents, where parameter sets the to be retrieved parameter or vector of parameters in theta, arm is the relevant integer index of the arm or vector of arms of interest, and the optional tail selects the last elements in the list. Returns a vector, matrix or array with the selected theta values.

print_data()

Prints a summary of the History log.

update_statistics()

Updates cumulative statistics.

get_agent_list()

Retrieve list of agents in History.

get_agent_count()

Retrieve number of agents in History.

get_simulation_count()

Retrieve number of simulations in History.

get_arm_choice_percentage(limit_agents)

Retrieve list of percentage arms chosen per agent for limit_agents.

get_meta_data()

Retrieve History meta data.

set_meta_datan(key, value, group = "sim", agent_name = NULL)

Set History meta data.

get_cumulative_data(limit_agents = NULL, limit_cols = NULL, interval = 1, cum_average = FALSE))

Retrieve cumulative statistics data.

get_cumulative_result(limit_agents = NULL, limit_cols = NULL, interval = 1, cum_average = FALSE))

Retrieve cumulative statistics data point.

save_theta_json(filename = "theta.json"))

Save theta in JSON format to a file. Warning: the theta log, and therefor the file, can get very large very fast.

get_theta(limit_agent, to_numeric_matrix = FALSE)

Retrieve an agent's simplified data.table version of the theta log. If to_numeric is TRUE, the data.table will be converted to a numeric matrix.

data

Active binding, read access to History's internal data.table.

cumulative

Active binding, read access to cumulative data by name through $ accessor.

meta

Active binding, read access to meta data by name through $ accessor.

See also

Core contextual classes: Bandit, Policy, Simulator, Agent, History, Plot

Bandit subclass examples: BasicBernoulliBandit, ContextualLogitBandit, OfflineReplayEvaluatorBandit

Policy subclass examples: EpsilonGreedyPolicy, ContextualLinTSPolicy

Examples

if (FALSE) {

  policy    <- EpsilonGreedyPolicy$new(epsilon = 0.1)
  bandit    <- BasicBernoulliBandit$new(weights = c(0.6, 0.1, 0.1))

  agent     <- Agent$new(policy, bandit, name = "E.G.", sparse = 0.5)

  history   <- Simulator$new(agents = agent,
                             horizon = 10,
                             simulations = 10)$run()

  summary(history)

  plot(history)

  dt <- history$get_data_table()

  df <- history$get_data_frame()

  print(history$cumulative$E.G.$cum_regret_sd)

  print(history$cumulative$E.G.$cum_regret)

}