# Markov Decision Processes (MDPs)

## Documentation for package ‘MDP2’ version 2.2.2.0

- DESCRIPTION file.
- User guides, package vignettes and other documentation.
- Package NEWS.

## Help Pages

binaryActionWriter | Function for writing actions of a HMDP model to binary files. The function defines sub-functions which can be used to define actions saved in a set of binary files. It is assumed that the states have been defined using 'binaryMDPWriter' and that the id of the states is known (can be retrieved using e.g. 'stateIdxDf').
binaryMDPWriter | Function for writing an HMDP model to binary files. The function defines sub-functions which can be used to define an HMDP model saved in a set of binary files.
convertBinary2HMP | Convert a HMDP model stored in binary format to a 'hmp' (XML) file. The function simply parse the binary files and create 'hmp' files using the 'hmpMDPWriter()'.
convertHMP2Binary | Convert a HMDP model stored in a hmp (xml) file to binary file format.
getBinInfoActions | Info about the actions in the HMDP model under consideration.
getBinInfoStates | Info about the states in the binary files of the HMDP model under consideration.
getHypergraph | Return the (parts of) state-expanded hypergraph
getInfo | Information about the MDP
getPolicy | Get parts of the optimal policy.
getRPO | Calculate the retention pay-off (RPO) or opportunity cost for some states.
getSteadyStatePr | Calculate the steady state transition probabilities for the founder process (level 0).
getWIdx | Return the index of a weight in the model. Note that index always start from zero (C++ style), i.e. the first weight, the first state at a stage etc has index 0.
hmpMDPWriter | Function for writing an HMDP model to a hmp file (XML). The function define sub-functions which can be used to define an HMDP model stored in a hmp file.
loadMDP | Load the HMDP model defined in the binary files. The model are created in memory using the external C++ library.
memoryMDPWriter | Function for building an HMDP model directly in memory.
plot.HMDP | Plot the state-expanded hypergraph of the MDP.
plotHypergraph | Plot parts of the state expanded hypergraph.
randomHMDP | Generate a "random" HMDP stored in a set of binary files.
runCalcWeights | Calculate weights based on current policy. Normally run after an optimal policy has been found.
runPolicyIteAve | Perform policy iteration using the average expected-weight Bellman operator on the MDP.
runPolicyIteDiscount | Perform policy iteration using the discounted expected-weight Bellman operator on the MDP.
runValueIte | Perform value iteration on the MDP.
saveMDP | Save the MDP to binary files
setPolicy | Modify the current policy by setting policy action of states.