:py:mod:`fragile.callbacks.tree` ================================ .. py:module:: fragile.callbacks.tree .. autoapi-nested-parse:: Data structure to keep track of the graph that stores all the states visited by a Swarm. Module Contents --------------- Classes ~~~~~~~ .. autoapisummary:: fragile.callbacks.tree.BaseTree fragile.callbacks.tree.NetworkxTree fragile.callbacks.tree.DataTree fragile.callbacks.tree.HistoryTree Functions ~~~~~~~~~ .. autoapisummary:: fragile.callbacks.tree.to_node_id Attributes ~~~~~~~~~~ .. autoapisummary:: fragile.callbacks.tree.DEFAULT_ROOT_ID .. py:data:: DEFAULT_ROOT_ID :annotation: = .. py:function:: to_node_id(x) .. py:class:: BaseTree(root_id=DEFAULT_ROOT_ID) Data structure in charge of storing the history of visited states of an algorithm run. .. py:method:: __call__(*args, **kwargs) Return the current instance of :class:`BaseTree`. This is used to avoid defining a ``tree_callable `` as ``lambda: tree_instance`` when initializing a :class:`Swarm`. If the :class:`BaseTree` needs is passed to a remote process, you may need to write custom serialization for it, or resort to creating an appropriate ``tree_callable``. .. py:method:: update(node_ids, parent_ids, n_iter = None, **kwargs) Update the history of the tree adding the necessary data to recreate a the trajectories sampled by the :class:`Swarm`. :param node_ids: List of states hashes representing the node_ids of the current states. :param parent_ids: List of states hashes representing the parent nodes of the current states. :param n_iter: Number of iteration of the algorithm when the data was sampled. :param kwargs: Keyword arguments representing different :class:`States` instances. :returns: None .. py:method:: reset(*args, **kwargs) Delete all the data currently stored and reset the internal state of the tree. .. py:method:: prune_tree(*args, **kwargs) Remove branches of the tree. .. py:class:: NetworkxTree(names = None, prune = False, root_id = DEFAULT_ROOT_ID, node_names = DEFAULT_NODE_DATA, edge_names = DEFAULT_EDGE_DATA, next_prefix = DEFAULT_NEXT_PREFIX) Bases: :py:obj:`BaseTree` It is a tree data structure that stores the paths followed by the walkers using a networkx DiGraph to keep track of the states relationships. .. py:attribute:: DEFAULT_ROOT_ID .. py:attribute:: DEFAULT_NEXT_PREFIX :annotation: = next_ .. py:attribute:: DEFAULT_NODE_DATA :annotation: = ['observs', 'rewards', 'oobs', 'scores'] .. py:attribute:: DEFAULT_EDGE_DATA :annotation: = ['actions'] .. py:method:: __len__() .. py:method:: __repr__() Return repr(self). .. py:method:: reset(root_id = None, state = None) Delete all the data currently stored and reset the internal state of the tree and reset the root node with the provided data. .. py:method:: update(node_ids, parent_ids, freeze_ids, freeze_parents, n_iter = None, **kwargs) Update the history of the tree adding the necessary data to recreate a the trajectories sampled by the :class:`Swarm`. :param node_ids: List of states hashes representing the node_ids of the current states. :param parent_ids: List of states hashes representing the parent nodes of the current states. :param freeze_ids: List of the ids of walkers that are marked as inactive. :param freeze_parents: List of the parent ids of walkers that are marked as inactive. :param n_iter: Number of iteration of the algorithm when the data was sampled. :param kwargs: Keyword arguments representing different :class:`States` instances. :returns: None .. py:method:: prune_tree(alive_leafs) Remove the branches that do not have a walker in their leaves. :param alive_leafs: Contains the ids of the leaf nodes that are being expanded by the walkers. :returns: None. .. py:method:: reset_graph(node_data, root_id = None, epoch = -1) Delete all the data currently stored and reset the internal state of the instance. :param root_id: The node id of the root node. :param node_data: Dictionary containing the data that will be added to the new node created. The keys of the dictionary contain the names of the attributes and its values the corresponding value added to the node. :param epoch: Epoch of the :class:`Swarm` run when the current node was generated. Defaults to -1. :returns: None. .. py:method:: append_leaf(leaf_id, parent_id, node_data, edge_data, epoch = -1) Add a new state as a leaf node of the tree to keep track of the trajectories of the swarm. :param leaf_id: Node id that identifies the new node that will be added to the tree. :param parent_id: Node id that identifies the parent of the node that will be added to the tree. :param node_data: Dictionary containing the data that will be added to the new node created. The keys of the dictionary contain the names of the attributes and its values the corresponding value added to the node. :param edge_data: Dictionary containing the data that will be added to the new edge created from parent_id to leaf_id. The keys of the dictionary contain the names of the attributes and its values the corresponding value added to the edge. :param epoch: Epoch of the :class:`Swarm` run when the current node was generated. Defaults to -1. :returns: None. .. py:method:: prune_dead_branches(dead_leafs, alive_leafs) Prune the orphan leaves that will no longer be used in order to save memory. :param dead_leafs: Leaves of the branches that will be removed. :param alive_leafs: Leaves of the branches that will kept being expanded. :returns: None .. py:method:: prune_branch(leaf_id, alive_nodes) Recursively prunes a branch that ends in an orphan leaf. :param leaf_id: Value that identifies the leaf of the tree. If `leaf_id` is the hash of a walker state `from_hash` needs to be `True`. Otherwise it refers to a node id of the leaf node. :param alive_nodes: Nodes of the branches that can be still expanded. :returns: None .. py:method:: get_parent(node_id) Get the node id of the parent of the target node. .. py:method:: get_branch(leaf_id) Return a list of the node ids of the path between root_node and leaf_id. :param leaf_id: Node id of the end of the path that will be returned. :returns: List containing the nodes found between the root node and ``leaf_id``. It starts with ``self.root_id`` and ends with ``leaf_id``. .. py:method:: get_path_node_ids(leaf_id, root_id = None) Get the data of the path between ``leaf_id`` and ``root_id``. :param leaf_id: Id that identifies the leaf of the tree. If ``leaf_id`` is the hash of a walker state ``from_hash`` needs to be ``True``. Otherwise it refers to a node id of the leaf node. :param root_id: Node id of the root node of the tree. :returns: List of the node ids between ``root`` and ``leaf_id``. .. py:method:: get_leaf_nodes() Return a list containing all the node ids of the leaves of the tree. .. py:method:: compose(other) Compose the graph of another :class:`BaseNetworkxTree` with the graph of the current instance. Composition is the simple union of the node sets and edge sets. The node sets of ``self.data`` and ``other.data`` do not need to be disjoint. The data in ``other`` takes precedence over the data in the current instance. :param other: Instance of :class:`BaseNetworkxTree` that will be composed with the current instance. :returns: None .. py:method:: _update_prune_sentinel() .. py:method:: _extract_data_from_states(index, state, only_node_data = False) .. py:class:: DataTree(names = None, prune = False, root_id = DEFAULT_ROOT_ID, node_names = NetworkxTree.DEFAULT_NODE_DATA, edge_names = NetworkxTree.DEFAULT_EDGE_DATA, next_prefix = NetworkxTree.DEFAULT_NEXT_PREFIX) Bases: :py:obj:`NetworkxTree` Tree data structure that keeps track of the visited states. It allows to save the :class:`Swarm` data after every iteration and methods to recover the sampled data after the algorithm run. The data that will be stored in the graph it's defined in the ``names`` parameter. For example: - If names is ``["observs", "actions"]``, the observations of every visited state will be stored as node attributes, and actions will be stored as edge attributes. - If names if ``["observs", "actions", "next_observs"]`` the same data will be stored, but when the data generator methods are called the observation corresponding to the next state will also be returned. The attribute ``names`` also defines the order of the data returned by the generator. As long as the data is stored in the graph (passing a valid ``names`` list at initialization, the order of the data can be redefined passing the ``names`` parameter to the generator method. For example, if the ``names`` passed at initialization is ``["states", "rewards"]``, you can call the generator methods with ``names=["rewards", "states", "next_states"]`` and the returned data will be a tuple containing (rewards, states, next_states). .. py:method:: _one_node_tuples(node, next_node, return_children) .. py:method:: _extract_one_node_data(data, names) Transform the one tuple of dictionaries returned by the data generators into a tuple of data values where each value corresponds to a name in names. .. py:method:: _process_batch(batch_data) Preprocess the list of tuples representing a batched group of elements and return a tuple of arrays representing the batched values for every data attribute. .. py:method:: _generate_batches(generator, names, batch_size = None) Return batches of processed data represented as tuples of arrays. .. py:method:: _validate_names(names) .. py:method:: path_data_generator(node_ids, return_children = False) Return a generator that returns the data corresponding to a path. Each value yielded corresponds to one element of the path. The edge data will be assigned to the parent node of the edge. This means the edge data contains the data associated with the transition to a child. The data corresponding to the last node of the path will not be returned. :param node_ids: Node ids of a path ordered. The first node of the path corresponds to the first element of ``node_ids``. :param return_children: If ``True`` the data corresponding to the child of each node in the path will also be returned. :Yields: tuple of dictionaries containing the data of each node and edge of the path, following the order of ``node_ids``. If ``return_children`` is ``False`` it will yield (node_data, edge_data). If ``return_children`` is ``True`` it will yield (node_data, edge_data, next_node_data). .. py:method:: random_nodes_generator(return_children = False) Return a generator that yields the data of all the nodes sampling them at random. Each value yielded corresponds to the data associated with one node and the edge connecting to one of its children. The edge data will be assigned to the parent node of the edge. This means the edge data of each node contains the data associated with the transition to a child. :param return_children: If ``True`` the data corresponding to the child of each node in the path will also be returned. :Yields: tuple of dictionaries containing the data for each node and edge sampled at random. If ``return_children`` is ``False`` it will yield (node_data, edge_data). If ``return_children`` is ``True`` it will yield (node_data, edge_data, next_node_data). .. py:method:: iterate_path_data(node_ids, batch_size = None, names = None) Return a generator that yields the data of the nodes contained in the provided path. :param node_ids: Ids of the nodes of the path that will be sampled. The nodes must be provided in order, starting with the first node of the path. :param batch_size: If it is not None, the generator will return batches of data with the target batch size. If Batch size is less than 1 it will return a single batch containing all the data. :param names: Names of the data attributes that will be yielded by the generator. :returns: Generator providing the data corresponding to a path in the internal tree. .. py:method:: iterate_nodes_at_random(batch_size = None, names = None) Return a generator that yields the data of the nodes contained in the provided path. :param batch_size: If it is not None, the generator will return batches of data with the target batch size. If Batch size is less than 1 it will return a single batch containing all the data. :param names: Names of the data attributes that will be yielded by the generator. :returns: Generator providing the data corresponding to all the nodes of the tree sampled at random. .. py:method:: iterate_branch(node_id, batch_size = None, names = None) Return a generator that yields the data of the nodes contained in the provided path. :param node_id: Ids of the last node of the branch that will be sampled. The first node will be the root node of the tree, and the last one will be the one with the provided ``node_id``. :param batch_size: If it is not None, the generator will return batches of data with the target batch size. If Batch size is less than 1 it will return a single batch containing all the data. :param names: Names of the data attributes that will be yielded by the generator. :returns: Generator providing the data corresponding to a branch of the internal tree. .. py:class:: HistoryTree(names = None, prune = False, root_id = DEFAULT_ROOT_ID, node_names = None, edge_names = ('actions', 'dt'), next_prefix = NetworkxTree.DEFAULT_NEXT_PREFIX, **kwargs) Bases: :py:obj:`fragile.callbacks.data_tracking.TrackWalkersId` Tree data structure that keeps track of the visited states. It allows to save the :class:`Swarm` data after every iteration and methods to recover the sampled data after the algorithm run. The data that will be stored in the graph it's defined in the ``names`` parameter. For example: - If names is ``["observs", "actions"]``, the observations of every visited state will be stored as node attributes, and actions will be stored as edge attributes. - If names if ``["observs", "actions", "next_observs"]`` the same data will be stored, but when the data generator methods are called the observation corresponding to the next state will also be returned. The attribute ``names`` also defines the order of the data returned by the generator. As long as the data is stored in the graph (passing a valid ``names`` list at initialization, the order of the data can be redefined passing the ``names`` parameter to the generator method. For example, if the ``names`` passed at initialization is ``["states", "rewards"]``, you can call the generator methods with ``names=["rewards", "states", "next_states"]`` and the returned data will be a tuple containing (rewards, states, next_states). .. py:attribute:: name :annotation: = tree .. py:method:: graph() :property: .. py:method:: data_tree() :property: .. py:method:: inputs() :property: Return a dictionary containing the data that this component needs to function. .. py:method:: __getattr__(item) .. py:method:: __repr__() Return repr(self). .. py:method:: to_html() .. py:method:: setup(swarm) Prepare the component during the setup phase of the :class:`Swarm`. .. py:method:: reset(root_id=None, state=None, **kwargs) Reset the internal state of the :class:`SwarmComponent`. :param inplace: Unused. Defaults to True. :type inplace: bool, optional :param root_walker: Set the internal state of the :class:`SwarmComponent` to this value. Defaults to None. :type root_walker: Optional[StateData], optional :param states: Set the internal state of the :class:`SwarmComponent` to this value. Defaults to None. :type states: Optional[StateData], optional :param kwargs: Other parameters required to reset the component. .. py:method:: update_tree() .. py:method:: before_walkers() .. py:method:: after_evolve() .. py:method:: after_reset() .. py:method:: iterate_root_path(batch_size = None, names = None) Return a generator that yields the data of the nodes contained in the provided path. :param batch_size: If it is not None, the generator will return batches of data with the target batch size. If Batch size is less than 1 it will return a single batch containing all the data. :param names: Names of the data attributes that will be yielded by the generator. :returns: Generator providing the data corresponding to a branch of the internal tree. .. py:method:: get_root_graph()