No notes for part 1, may have useful definitions? (revisit if necessary)
Intelligent Autonomous Agents
AI Agent
An agent is a unit that makes an intelligent decision given the opportunity, it uses the knowledge it has at that point in time to make a decision. Decision is limited by range of actions possible at that point in time.
Link to original
For intelligent agents, we consider its environment, sensors and effectors / actuators.
Wooldridge & Jennings (1995), defines agents as:
- proactive, try to achieve specific goals
- reactive, they respond to their perceptions of their environment
- social, they are aware of other entities in their environment
- autonomous, to a greater or lesser extent
Constraints on decisions
- Option Uncertainty: we may not have information on possible alternative options
- Incommensurability: we may not be able to compare alternative options
- Outcome uncertainty: we may not know what happens after we execute our selected option
- Resource constraints: some problems are computationally very hard
- Time constraints: we usually have a fixed time to make a decision
Rational agent
Rational Agent
A rational agent makes decisions based on knowledge it has (agent acting in the real world under resource constraints).
An ideal rational agent will act to “maximise its expected performance measure” on the basis of provided information by ‘percepts’ from the environment. However rational doesn’t mean perfect or omniscient.
Link to original
Models of agents
A simple agent maps each percept directly to an action. We can only make a correct decision based on the current percept.
We can introduce new percepts in the form of existing knowledge about the state, as well as use knowledge about the world to model inaccessible parts of the environment.
Utility
Utility
Utility is a quantitative value that measures the benefit or value of that state to a particular agent. It provides a way for us to compare different actions but in reality is not trivial to calculate.
Link to original
Expected Utility
Most states are not certain, may only occur with some probability. Hence if we can calculate utility and probability, we can find Expected Utility of the state:
For (set of states), and where indicates the sum over all states in :
We run into issues with utilities such as:
Link to original
- summing up over all states might be very costly in terms of processing resource / time
- not always straightforward to calculate the utility
- difficult to have a long term view
Types of Environments
Accessible (environment)
An environment is accessible if the agent can obtain accurate and up-to-date information about all the relevant aspects.
Link to original
Deterministic (environment)
A deterministic environment is one in which any action has a single guaranteed effect - no uncertainty.
Link to original
Stochastic (environment)
An environment where we can quantify the non-determinism with probability is called stochastic.
Link to original
Episodic (environment)
An episodic environment is where the performance of an agent is dependent on a number of discrete episodes, with no link between the performance of an agent in different episodes.
Link to original
Non-episodic (environment)
In a non-episodic environment, the current decision affects future decisions.
Link to original
Dynamic (environment)
A dynamic environment has other processes operating on it and may change in ways beyond the agent’s control.
Link to original
Static (environment)
A static environment remains unchanged except by the performance of the agent’s own actions.
Link to original
Discrete (environment)
If the agent’s environment has a fixed, finite number of actions and percepts then we call it discrete, otherwise we call it continuous.
Link to original
Run (environment)
A run is a sequence of interleaved states and actions by an agent acting in an environment.
Link to original