diff --git a/docs/4-SEARCH-IN-COMPLEX-ENVIRONMENTS.md b/docs/4-SEARCH-IN-COMPLEX-ENVIRONMENTS.md new file mode 100644 index 0000000..f9929e5 --- /dev/null +++ b/docs/4-SEARCH-IN-COMPLEX-ENVIRONMENTS.md @@ -0,0 +1,416 @@ +# Search in complex Environments + +## Local Search + +Differently from a path search, they do not store steps taken. This is because we need only the **final state** +and no more. + +However this comes at the cost of not being systematic, thus not exploring all the potential space, however +having an upper hand on memory and computation in infinte or large spaces + +> [!TIP] +> These are good when we only care about the final state, not the process of achieving it. +> Moreover it uses little memory and can be used for large spaces where conventional search fails + +> [!CAUTION] +> They are not systematic and may never explore space where the solution resides + +## Optimization problems + +We need to find a place in a function where we find a **reasonable highest number** (or lowest). +Usually our function has: + +- Local Maxima +- Ridges: Places where we have many local maxima near to each other but are not conected to each other (difficult to navigate with greedy search) +- Plateaus: Places where the difference in height is minimal or nonexistant + +We have some algorithms and strategies: + +- Stochastic Hill climbing: Choose a higher point with some probability according to its height +- First Choice hill climbing: Generate successor nodes randombly until one is better than the current (implements Stochastic Hill Climbing) +- Random-Restart Hill Climbing: If you fail, try again + +> [!NOTE] +> Computing expected number of steps in a Random Restart: +> +> - $p$: Probability of success of the algorithm without restart +> - $S$: Number of steps of a **successful** iteration +> - $s$: Number of steps of an **unsuccessful** iteration +> - $\frac{1}{p}$: Expected number of retries before finding a solution +> +> $\text{Expected Steps} = 1 \cdot S + \frac{1-p}{p} \cdot s$ + +### Simulated Annealing + +We start searching for our objective, however we initially search with +a stat called **Temperature** which allow the model to take more random choices +the higher it is. + +If it is not better than our current state, it'll use **Boltzmann distribution** ($e^{\frac{\Delta E}{T}}$) +across all $\Delta E$ of successor nodes. + +```python +def simulated_annealing(problem, schedule) -> State: + + # Take initial state, time, temperature + current = problem.initial_state + start_time = time.now() + T = schedule(0) + + # While temperature is higher than 0 + while (T > 0): + + # Here I generate successors, but can + # be implemented in other ways + successors : list[State] = current.children + move = random.int(0, len(successors)) + candidate = successors[move] + + # Compute the fitness of the move + delta_energy = candidate.value - current.value + + if delta_energy > 0: + current = candidate + else: + # Notice that since the energy is negative, probability is less than 1 + # so, when T -> 0, it'll be e^(-inf) == 0 + boltzman_probability = e^(delta_energy / T) + # Probability helper + if random() < boltzman_probability: + current = candidate + + # Update temperature + T = schedule(time.now() - start_time) + + return current + + + +``` + +> [!NOTE] +> The lower the temperature, the higher the concentration over a local maximum. + +> [!TIP] +> Low memory usage and fast computation + +> [!CAUTION] +> Suboptimal solutions with respect to other algorithms + +### Local Beam Search + +- Generate $k$ successors of all $k$ states kept in memory +- If one is goal, stop +- Else keeps the best $k$ in memory and starts again + +```python +def local_beam_search(problem, k: int) -> State: + + # Get k initial states + current: problem.random_states(k) + halt = False + + while (not halt): + + successors = [] + + # Note: this may be parallelizable + + # Add all successors + for state in current: + + children = state.children + + # check if a children is a goal + for child in children: + + # Found, stop execution + if is_goal(child): + return child # If parallelized, no return + + successors.push( + state.children + ) + + # Note: this is sequential, unless we want to implement + # a distributed heap sort + current = successors.sort()[0:k] + +``` + +> [!TIP] +> Better use of available memory and fast computation (parallelizable) + +> [!CAUTION] +> Lack of diversity in its $k$ nodes + +> [!NOTE] +> Stochastic Beam Search takes $k$ nodes with a probability proportional to successor's value + +### Evolutionary algorithms + +pg 133 to 137 + +Similar to **Stochastic Beam Search**. + +They have the following properties: + +- Population size +- Indivisual Representation + - Boolean String (Genetic Algorithms) + - Real numbers (Evolution Strategies) + - Computer Program (Genetic Programming) +- Mixing number $\rho$ (usually $\rho = 2$): Number of parents per offspring +- Selection Process + - Select n fittest + - Select n fittest and $\rho$ fittest as parents +- Recombination Procedure: Randomly select a **Crossover point**, split there and then recombine +- Mutation Rate: Probability of an offspring to have a spontaneous mutation +- Making the next generation + - Only children + - Mix Children and Fittest Parents + +> [!NOTE] +> In Genetic Algorithms we usually talk about: +> +> - Schema: String with incomplete cells (eg: 1*98**00) +> - Instance: String that has schema and complete cells (eg: 12980000 - 13987500) +> +> The more a Schema is fittest, the more istances of that schema will appear over time + +```python + +# This is somewhat a pseudoimplementation and +# many details must be implemented +def genetic_evolution( + problem, + population, + fitness, + breed, + mutate, + mutation_probability, + max_steps +) + + population : list[(State, float)]= [] + + # We are assuming different initial states + for i in range(0, population) + candidate = problem.random_unique_state() + + if is_goal(candidate): + return candidate + + population.push(candidate, fitness(candidate)) + + population = population.sort() + + halt = False + step = 0 + + while (not halt): + step += 1 + + # Create new generation + new_generation = breed(population) + + # Mutate kids + for kid in new_generation: + kid = mutate(kid, mutation_probability) + + # Check if kid is better than parents + for kid in new_generation: + + kid_score = fitness(kid) + + if is_goal(kid): + return kid + + # If kid is better than last of population, + # kid is enough fit + if kid_score > population[-1].score: + + # Remove least fit entity, + # replace with kid and SORT + # (unless you used a heap) + population.pop() + population.push( + (kid, kid_score) + ) + population.sort() + + if step > max_steps: + halt = True + + return population[0] + +``` + +## Local Search in Continuous Spaces + +We have these techniques: + +- Gradient Descent +- Discretize +- Newton Raphson + +## Search with non-deterministic actions + +Since our agents **plan** ahead ,it must know whatever will happen after an action. + +> [!NOTE] +> Brief recap +> +> - **Deterministic**: $result(state_{i_1}, action_j) = state_{i_2}$ +> - **Non-Determinsitic**: $result(state_{i_1}, action_j) = \Set{ \cup_k \; state_{k}}$ +> +> It can be seen that in a Non-Deterministic world, we have a **set of states resulting from an action** + +If we would be in our agent's shoes for a moment, we would plan accordingly, saying that **if** we will be in $state_a$ **then** we'll act +in a way, **else** in another. + +Basically, we are finding a plan that involves all possible possibilities and then we delay the moment of acting when we'll have more info, +discarding plans for which conditions were not met. + +In our agent world it translates roughly to **conditional plans** (or contingency plans). +Our plan is not anymore a sequence of actions, but rather contains some **if-else** or **switch-case** actions. + +```python +# This is just a pseudo implementation +# and it's supposed to be used only +# on a plan, not while planning +class ConditionalAction(Action): + + def __init__( + self, + conditional_map: dict[State, Action] + ): + self.__conditional_map = conditional_map + + + def decide(self, percept: Percept): + state = get_state(percept) + return conditional_map[state] +``` + +### AND - OR Search Trees + +These trees have the following properties: + +- **OR Nodes**: Nodes resulting from a choice of an action +- **AND Nodes**: Nodes resulting from having multiple states coming from our action + +> [!NOTE] +> We may think of AND Nodes as **meta states** or **shrodinger states**, where they are all technically real +> and not at the same time, until we observe. +> +> Contrary to the exploration for OR Nodes, we are **forced** to explore all AND Nodes, as we need to plan +> for them in case they exist. + +> [!WARNING] +> If any AND Node has a **failure**, then we fail + +> [!TIP] +> We will succeed only if all AND Nodes return a success + +Some search techniques do not implement a retry action if there's a failure in an action, so that they can +avoid cycles. Retrying needs having cycles and this calls for the need of a **while-action** + +```python + +class RetryAction(Action): + + def __init__( + self, + stuck_state: State, + action: Action + ): + self.__stuck_state = stuck_state + self.__action + + + def act(self, percept: Percept): + current_state = get_state(percept) + + while (current_state == self.__stuck_state): + current_state = result(current_state, self.__action) + + return current_state + +``` + +## Search in Partially Observable Environments + +This is a situation where you are unsure about a state you are in. In this case you know the rules, so you know the outcome or outcomes, +however you do not know where you are. + +If you were blind folded, you would have to assess where you are and proceed from there on. But imagine that you had some pits in your home, +in that case you would not wander aimelessly, but rather check each step you do. + +### No observation (Sensorless Problem) + +This is an extreme version of the preceeding problem. Here you can't perceive anything at all, so you can't narrow down states according +to percepts, but only by acting. + +Imagine that moving is always an option even if blocked and you know exactly the geometry of the room. In this case, by acting in a certain way, +you can always certainly know that you'll come to a certain point. This is called **coercion**: The ability to make a belief state to converge to a +single state. + +Since we can coerce states and we have no percepts, there are **no contingency plans** and **solutions are sequence of actions** + +> [!TIP] +> A sensorless plan is not useless, and sometimes is better when you are resource constraint, as it may be faster + +- **Initial State**: Set of all states ($\mathcal{S}$) +- **Number of Belief States**: $2^{\mathcal{S}}$[^number-of-subsets] +- **Legal Actions** + - **If any action is legal in any state**: All actions that can be performed in any state contained in the belief state $\cup_{s \in b}a_{\mathcal{P}}(s)$ + - **If actions may be not legal in some state**: Only actions that are legal in **all** states of the belief state $\cap_{s \in b}a_{\mathcal{P}}(s)$ +- **Transition Model** + - **If Actions are deterministic**: Our belief state is the **Set** of all states coming from actions and it'll have a size equal or less than our previous belief state + - **If Actions are not deterministic**: Our belief state is the **Set** of all states coming from actions and can be of a dimension higher than our previosu belief state +- **Goal** + - **Possibly Achievable**: One of the states in the belief state is a goal state + - **Necessarily Achievable**: All of states in the belief state are a goal state +- **Cost** + - **Uniform** + - **Variable** + +#### Pruning + +Whenever we encounter a superset of a belief state we already encountered, we can prune it. +This comes from the fact that a superset is a much harder problem than a subset, and since it would probably include also our +subset solution, then this means that it's easier to directly explore our subset. + +On the contrary, let's say that we already found a solution for our superset, then we also found a solution for the subset, +proving it to be solvable + +#### Incremental Searching + +Instead of searching a solution for each state in the belief state **contemporarily**, we can search a solution for just one of +those states and check if it goes well for the others, and if not, search again. + +That comes from the fact that all states must have the same sequence of actions, so if we run out of possible solutions for a state, +then our problem is unsolvable. + +### Partial Observability + +Now, we can observe at least a portion of the belief state we are in, so it'll be easier to construct and coerce our belief state into a single state + +Here we do the following: + +1. Check my environment (get initial state) +2. Choose to do an action and predict next states +3. Check what states I would find after checking that state (Here we will have disjoint states if sensing is deterministic, either one or the other) +4. Repeat from step 2 until here until goal is reached + +As for [Non deterministic](#search-with-non-deterministic-actions) actions, ww need to explore each possible state as we won't +be sure of where we are until we execute our plan + +## Online Searching + +Book from pg. 152 to 159 + + + +[^number-of-subsets]: [Proof](https://math.stackexchange.com/questions/546414/what-is-the-proof-that-the-total-number-of-subsets-of-a-set-is-2n) \ No newline at end of file diff --git a/docs/5-CONSTRAINT-SATISFACTION-PROBLEMS.md b/docs/5-CONSTRAINT-SATISFACTION-PROBLEMS.md new file mode 100644 index 0000000..d35d609 --- /dev/null +++ b/docs/5-CONSTRAINT-SATISFACTION-PROBLEMS.md @@ -0,0 +1,99 @@ +# Constraint Satisfaction Problems + +Book pg. 164 to 169 + +We have 3 components + +- $\mathcal{X}$: Set of variables +- $\mathcal{D}$: Set of domains, one (domain) for each variable +- $\mathcal{C}$: Set of constraints + +Here we try to assign a value to variables, such that each variable holds a value contained in their +domain, without violating any constraint. + +A solution to this problem must be: + +- Consistent: Each assignment is legal +- Complete: All variables have a value assigned + +## Inference in CSP + +We can inference a solution in these ways + +- Generate successors and if not consistent assignments, prune and continue +- Constraint Propagation:\ + Use contraints to reduce possible assignments and mix with search, if needed + +The idea is to treat each assignment as a **vertex** and each contraint as an **edge**, we our tree would not have inconsistent values, making +the search faster. + +## Consistency + +### Node consistency + +A domain already satisfies constraints + +### Arc Consistency + +A variable is arc consistent wrt another variable id its domain respects all constraints + +> [!WARNING] +> Arco Consistency is not useful for every problem + +> [!NOTE] +> AC-3 is an algorithm to achieve Arc Consistency (alorithm on page 171 book) + +### Path Consistency + +It is achievable on 3 variables and says that a set of 2 variable is path consistent wrt a 3rd only if for each assignment +consistent with constraints between these variables makes it possible to make a legal assignment on the 3rd + +### K-Consistency + +basically the same as the one above, but for k-variables, technically we need $O(n^2 d)$ to check if a problem is +total k-consistent (consistent from 1 up to k), however it is expensive and only 2-Consistency is usually computed. + +## Global Contraints + +Constraints that take an arbitrary number of variables (but not necessarily all). + +### Checking for Aldiff Constraint + +Aldiff constraint says that all variables must have different values. + +A way to check for inconsistencies is to remove variables with single value domains and remove such value +from all domains, then iterate until there are no more single value domain variables. + +If an empty domain is produced, or there are more variables than domain values, congrats!! You found an inconsistency + +### Checking for Atmost Constraint + +Atmost constraint says that we can assign max tot resources + +Sum all minimum resources for each problem, and if it goes over the max, we have an inconsistency + +### Bound Consistency + +We check that for each variable X and for both the lower and higher bound of X there exists a value of U that satisfies the constraints between X and each Y. + +## Backtracking Search in CSP + +Book pg 175 to 178 + +After finishing the constraint propagation, reducing our options, we still need to search for a solution. + +- The order of values is not relevant (we can save up on computation and memoru) + +### Forward Checking + + + +### Intelligent Backtracking (Backjumping) + + + +## The structure of Problems + + + +Book from 183 to 187 diff --git a/docs/7-Logical-Agents.md b/docs/7-Logical-Agents.md new file mode 100644 index 0000000..59419dd --- /dev/null +++ b/docs/7-Logical-Agents.md @@ -0,0 +1,17 @@ +# Logical agents + +## Inference Algorithms + +they have these properties: + +- Sounding or Truth-preserving: finds only real entailed sentences +- Completeness: Same as Searching in Ch 3 +- Grounding: How can be sure that our knowledge is real + +## Sentences + +They have the following properties: + +- logical equivalence: These sentences are logically the same +- Validity: A sentence is valid if equivalent to true sentence (True in all worlds) +- Satisfiability: True in **some** worlds