Foundation-of-Artificial-Intelligence/docs/2-INTELLIGENT-AGENTS.md

Intelligent Agents

Task Environment

Performance

How well the job is done

Environment

Something we must accept as it is

Actuators

Something we can use to change our Environment

Sensors

Something we can use to perceive our Environment

Properties of a Task Environment

• Observability
  • Fully Observable: Sensors gather all info
  • Partially Observable: Sensors gather info, but some is unavailable
  • Unobservable: No Sensors at all
• Number of Agents:
  • Multi-Agent: Other Agents (even people may be considered agents) live in the environment

    Note

    In order to be considered an agent, the other entity must maximize an objective that depends on our agent behaviour

    • Cooperative: All agents try to maximize the same objective
    • Competitive: Agent objective can be maximized penalizing other agents objective
  • Single-Agent: Only one agent exists
• Predictability
  • Deterministic: We can predict everything
  • Stochastic: We can predict outcomes according to some probabilities
  • Nondeterministic: We cannot predict everything nor the probability
• Memory-Dependent
  • Episodic: Each stimulus-action is independent from previous actions
  • Sequential: Current actions may influence ones in the future, so we need to keep memory
• Staticity
  • Static: Environment does not change while our agent is deciding
  • Dynamic: Environment changes while our agent is deliberating
  • SemiDynamic: Environment is Static, but agent's performance changes with time
• Continuousity (Applies to States)
  • Continuous: State has continuous elements
  • Discrete: State has no continuous elements
• Knowlegde

  Caution

  It is not influenced by Observability, as this refers to the outcomes of actions and not over the state of the agent

  • Known: Each rule is known a priori (known outcomes)
  • Unknown: The agent must discover environment rules (unknown outcomes)

Environment classes

According to properties, we can define a class of Environments on where we can test our agents

Architecture

The actual hardware (actuators) where our program will run, like a robot, or a pc.

Agent Programs

Note

All programs have the same pseudo code:

 def agent(percept) -> Action:  

Table Driven Agent

It basically has all possible reactions to stimuli at time \mathcal{T}_i, thus a space of \sum_{t=1}^{T}|\mathcal{S}|^{t}, which quicly becomes enormous

Tip

It is actually complete and makes us react at best

Caution

It is very memory consuming, so it is suitable for small problems

class TableDrivenAgent(Agent):

    def __init__(
        self,
        action_table: dict[list[Percept], Action]
    ):
        self.__action_table = action_table
        self.__percept_sequence : list[Percept]= []

    
    def agent(self, percept: Percept) -> Action:
        self.__percept_sequence.push(
            percept
        )

        return self.__action_table.get(
            self.__percept_sequence
        )

Reflex Agent

Acts only based on stimuli at that very time t

Tip

It only reacts to stimuli based on the current state, so it's smaller and very fast

Caution

It is very limited in its capabilities and requires a fully observable environment

class ReflexAgent(Agent):

    def __init__(
        self,
        rules: list[Rule], # It depends on how the actual function is implemented
    ):
        self.__rules = rules
    

    def agent(self, percept: Percept) -> Action:
        status = self.__get_status(percept)
        rule = self.__get_rule(status)
        return rule.action


    # MUST BE IMPLEMENTED
    def __get_status(self, percept: Percept) -> Status:
        pass


    # MUST BE IMPLENTED (it uses our rules)
    def __get_rule(self, state: State) -> Rule:
        pass

Model-Based Reflex Agent

This agent has 2 models that helps it keeping an internal representation of the world:

• Transition Model:
  Knowledge of how the world works (What my actions do)
• Sensor Model:
  Knowledge of how the world is reflected in agent's percept (How the world evolves without me)

Tip

It only reacts to stimuli based on the current state, but is also capable of predicting next state, thus having some info on unobserved states

Caution

It is more complicated to code and slightly worse in terms of raw speed. While it is more flexible, it is still somewhat limited

class ReflexAgent(Agent):

    def __init__(
        self,
        initial_state: State,
        transition_model: TransitionModel,  # Implementation dependent
        sensor_model: SensorModel,          # Implementation dependent
        rules: Rules                        # Implementation dependent
    ):
        self.__current_state = initial_state
        self.__transition_model = transition_model
        self.__sensor_model = sensor_model
        self.__rules = rules
        self.__last_action : Action = None
    

    def agent(self, percept: Percept) -> Action:
        self.__update_state(percept)
        rule = self.__get_rule()
        return rule.action


    # MUST BE IMPLEMENTED
    def __update_state(self, percept: Percept) -> State:
        """
        Uses:
            - percept
            - self.__current_state,
            - self.__last_action,
            - self.__transiton_model,
            - self.__sensor_model
        """
        # Do something
        self.__current_state = state


    # MUST BE IMPLENTED (it uses our rules)
    def __get_rule(self) -> Rule:
        """
        Uses:
            self.__current_state,
            self.__rules
        """
        # Do something
        return rule

Goal-Based Agent

It's an agent that has an internal state representation, can predict the next state and chooses an action that satisfies its goals.

Tip

It is very flexible and needs less hardcoded info, compared to a reflex-based approach, allowing a rapid change of goals without reprogramming the agent

Caution

It is more computationally expensive to implement, moreover it requires a strategy to choose the best action

Utility-Based Agent

It's an agent that performs to maximize its expected utility, useful when goals are incompatible and there's uncertainty aboout how to achieve a goal

Tip

Useful when we have many goals with different importance and when we need to balance some incompatible ones

Caution

It adds another layer of computation and since it chooses based on estimantions, it could be wrong

Note

Not each goal-based agent has a model to guide it

Learning Agent

Each agent may be a Learning Agent. and it is composed of:

• Learning Element: Responsible for improvements
• Performance Element: The entire agent, which takes actions
• Critic: Gives Feedback to the Learning element on how to improve the agent
• Problem Genrator: Suggests new actions to promote exploration of possibilites, avoiding to repeat the best known action

States Representation

Note

Read pg 76 to 78 of Artificial Intelligence: A Modern Approach