diff --git a/Chapters/12-Transformers/INDEX.md b/Chapters/12-Transformers/INDEX.md index e69de29..c680432 100644 --- a/Chapters/12-Transformers/INDEX.md +++ b/Chapters/12-Transformers/INDEX.md @@ -0,0 +1,187 @@ +# Transformers + +## Block Components + +The idea is that each Transformer block is made of the **same number** of [`Encoders`](#encoder) and +[`Decoders`](#decoder) + +![image](./Images/PNGs/transformer-high-level.png) + +> [!NOTE] +> Input and output are vectors of **fixed size** with padding + +Before feeding our input, we split and embed each word into a fixed vector size. This size depends on the length of +longest sentence in our training set + +### Embedder + +While this is not a real component per se, this is the first phase before even coming +to the first `encoder` and `decoder`. + +Here we transform each word of the input into an ***embedding*** and add a vector to account for +position. This positional encoding can either be learnt or can follow this formula: + + +- Even size: +$$ +\text{positional\_encoding}_{ + (position, 2\text{size}) +} = \sin\left( + \frac{ + pos + }{ + 10000^{ + \frac{ + 2\text{size} + }{ + \text{model\_depth} + } + } + } + \right) +$$ +- Odd size: +$$ +\text{positional\_encoding}_{ + (position, 2\text{size} + 1) +} = \cos\left( + \frac{ + pos + }{ + 10000^{ + \frac{ + 2\text{size} + }{ + \text{model\_depth} + } + } + } + \right) +$$ + + +### Encoder + +> [!CAUTION] +> Weights are not shared between `encoders` or `decoders` + +Each phase happens for each word. In other words, if our embed size is 512, we have 512 `Self Attentions` and +512 `Feed Forward NN` **per `encoder`** + +![Image](./Images/PNGs/encoder.png) + +#### Encoder Self Attention + +> [!WARNING] +> This step is the most expensive one as it involves many computations + +Self Attention is a step in which each ***token*** gets the knowledge of previous ones. + +During this step, we produce 3 vectors that are **usually smaller**, for example 64 instead of 512: + +- **Queries** $\rightarrow q_{i}$ +- **Keys** $\rightarrow k_{i}$ +- **Values** $\rightarrow v_{i}$ + +We use these values to compute a **score** that will tell us **how much to focus on certain parts of the sentence +while encoding a token** + +In order to compute the final encoding we do these for each encoding word $i$: + +- Compute score for each word $j$ : $\text{score}_{j} = q_{i} \cdot k_{j}$ +- Divide each score by the square root of the size of these *helping vectors*: + $\text{score}_{j} = \frac{\text{score}_{j}}{\sqrt{\text{size}}}$ +- Compute softmax of all scores +- Multiply softmax each score per its value: $\text{score}_{j} = \text{score}_{j} \cdot v_{j}$ +- Sum them all: $\text{encoding}_{i} = \sum_{j}^{N} \text{score}_{j}$ + +> [!NOTE] +> These steps will be done with matrices, not in this sequential way + +##### Multi-Headed Attention + +Instead of doing the Attention operation once, we do it more times, by having differente matrices to produce +our *helping vectors*. + +This produces N encodings for each ***token***, or N matrices of encodings. + +The trick here is to **concatenate all encoding matrices** and **learn a new weight matrix** that will +**combine them** + +#### Residuals + +In order no to lose some information along the path, after each `Feed Forward` and `Self-Attention` +we add inputs to each ***sublayer*** `outputs` and we do a `Layer Normalization` + +#### Encoder Feed Forward NN + +> [!TIP] +> This step is mostly **parallel** as there's no dependency between *neighbour vectors* + +### Decoder + +> [!NOTE] +> The decoding phase is slower than the encoding one, as it is sequential, producing a token for each iteration. +> However it can be sped up by producing several tokerns at once + +After the **last `Encoder`** has produced its `output`, $K$ and $V$ vectors, these are then used by +**all `Decoders`** during their self attention step, meaning they are **shared** among all `Decoders`. + +All `Decoders` steps are then **repeated until we get a `` token** which will tell the decoder to stop. + +#### Decoder Self Attention + +It's almost the same as in the `encoding` phase, though here, since we have no future `outputs`, we can only take into +account only previous ***tokens***, by setting future ones to `-inf`. +Moreover, here the `Key` and `Values` Mappings come from the `encoder` pase, while the +`Queue` Mapping is learnt here. + +#### Final Steps + +##### Linear Layer + +Produces a vector of ***logits***, one per each ***known words***. + +##### Softmax Layer + +We then score these ***logits*** over a `SoftMax` to get probabilities. We then take the highest one, usually. + +If we implement ***Temperature***, though, we can take some `tokens` that are less probable, but having less predictability and +have some results that feel more natural. + +## Training a Transformer + + + +## Known Transformers + +### BERT (Bidirectional Encoder Representations from Transformers) + +Differently from other `Transformers`, it uses only `Encoder` blocks. + +It can be used as a classifier and can be fine tuned. + +The fine tuning happens by **masking** input and **predict** the **masked word**: + +- 15% of total words in input are masked + - 80% will become a `[masked]` token + - 10% will become random words + - 10% will remain unchanged + +#### Bert tasks + +- **Classification** +- **Fine Tuning** +- **2 sentences tasks** + - **Are they paraphrases?** + - **Does one sentence follow from this other one?** +- **Feature Extraction**: "Allows us to extract feature to use in our model + +### GPT-2 + +Differently from other `Transformers`, it uses only `Decoder` blocks. + +Since it has no `encoders`, `GPT-2` takes `outputs` and append them to the original `input`. This is called **autoregression**. +This, however, limits `GPT-2` on how to learn context on `input` because of `masking`. + +During `evaluation`, `GPT-2` does not recompute `V`, `K` and `Q` for previous tokens, but hold on their previosu values. diff --git a/Chapters/12-Transformers/Images/Excalidraw/encoder.excalidraw.json b/Chapters/12-Transformers/Images/Excalidraw/encoder.excalidraw.json new file mode 100644 index 0000000..04fe38d --- /dev/null +++ b/Chapters/12-Transformers/Images/Excalidraw/encoder.excalidraw.json @@ -0,0 +1,2055 @@ +{ + "type": "excalidraw", + "version": 2, + "source": "https://marketplace.visualstudio.com/items?itemName=pomdtr.excalidraw-editor", + "elements": [ + { + "id": "1kSrI1SjU3TIAyt7I9hO3", + "type": "rectangle", + "x": 293.5, + "y": 52, + "width": 970.5, + "height": 396, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#b2f2bb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "a0", + "roundness": { + "type": 3 + }, + "seed": 938815260, + "version": 200, + "versionNonce": 780530844, + "isDeleted": false, + "boundElements": null, + "updated": 1756974376822, + "link": null, + "locked": false + }, + { + "id": "TQYWmwiNthe00FnHfb90S", + "type": "text", + "x": 313.5, + "y": 68.5, + "width": 76.99989318847656, + "height": 25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "a1", + "roundness": null, + "seed": 1532751900, + "version": 351, + "versionNonce": 508793892, + "isDeleted": false, + "boundElements": null, + "updated": 1756974380821, + "link": null, + "locked": false, + "text": "Encoder", + "fontSize": 20, + "fontFamily": 8, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Encoder", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "id": "fx_clfKT5QAuSTqc0JKNa", + "type": "rectangle", + "x": 415, + "y": 470.5, + "width": 176.5, + "height": 49.5, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#a5d8ff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "a2O", + "roundness": { + "type": 3 + }, + "seed": 259831588, + "version": 263, + "versionNonce": 1726026652, + "isDeleted": false, + "boundElements": [ + { + "type": "text", + "id": "Zie2bKIu-I-dcvQJuDEZ_" + } + ], + "updated": 1756974514865, + "link": null, + "locked": false + }, + { + "id": "Zie2bKIu-I-dcvQJuDEZ_", + "type": "text", + "x": 492.25001525878906, + "y": 482.75, + "width": 21.999969482421875, + "height": 25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#9775fa", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "a2S", + "roundness": null, + "seed": 919025828, + "version": 4, + "versionNonce": 1642317476, + "isDeleted": false, + "boundElements": null, + "updated": 1756974518214, + "link": null, + "locked": false, + "text": "X1", + "fontSize": 20, + "fontFamily": 8, + "textAlign": "center", + "verticalAlign": "middle", + "containerId": "fx_clfKT5QAuSTqc0JKNa", + "originalText": "X1", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "id": "3usaxbyef2wUj6m158aVY", + "type": "rectangle", + "x": 637, + "y": 470.5, + "width": 176.5, + "height": 49.5, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#a5d8ff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "a2W", + "roundness": { + "type": 3 + }, + "seed": 346174116, + "version": 530, + "versionNonce": 24948380, + "isDeleted": false, + "boundElements": [ + { + "type": "text", + "id": "8krIvGUkLZEHojr7bQbYy" + } + ], + "updated": 1756974519893, + "link": null, + "locked": false + }, + { + "id": "8krIvGUkLZEHojr7bQbYy", + "type": "text", + "x": 714.2500152587891, + "y": 482.75, + "width": 21.999969482421875, + "height": 25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#9775fa", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "a2a", + "roundness": null, + "seed": 924151204, + "version": 4, + "versionNonce": 1391426468, + "isDeleted": false, + "boundElements": null, + "updated": 1756974520871, + "link": null, + "locked": false, + "text": "X2", + "fontSize": 20, + "fontFamily": 8, + "textAlign": "center", + "verticalAlign": "middle", + "containerId": "3usaxbyef2wUj6m158aVY", + "originalText": "X2", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "id": "PaL3txItBxSGt3VCDyN6e", + "type": "rectangle", + "x": 1052, + "y": 470.5, + "width": 176.5, + "height": 49.5, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#a5d8ff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "a3", + "roundness": { + "type": 3 + }, + "seed": 1586249252, + "version": 650, + "versionNonce": 1198000540, + "isDeleted": false, + "boundElements": [ + { + "type": "text", + "id": "wZ0bdoDWvN8bcHXuQ0QrV" + } + ], + "updated": 1756974522606, + "link": null, + "locked": false + }, + { + "id": "wZ0bdoDWvN8bcHXuQ0QrV", + "type": "text", + "x": 1129.250015258789, + "y": 482.75, + "width": 21.999969482421875, + "height": 25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#9775fa", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "a3G", + "roundness": null, + "seed": 1719132836, + "version": 4, + "versionNonce": 653226148, + "isDeleted": false, + "boundElements": null, + "updated": 1756974524694, + "link": null, + "locked": false, + "text": "XN", + "fontSize": 20, + "fontFamily": 8, + "textAlign": "center", + "verticalAlign": "middle", + "containerId": "PaL3txItBxSGt3VCDyN6e", + "originalText": "XN", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "id": "DydO-ttPWT2eVOChqiWK-", + "type": "line", + "x": 830, + "y": 492.5, + "width": 202, + "height": 0, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#a5d8ff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "a4", + "roundness": { + "type": 2 + }, + "seed": 103340452, + "version": 199, + "versionNonce": 1889927836, + "isDeleted": false, + "boundElements": null, + "updated": 1756974270948, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 202, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": null + }, + { + "id": "UmzVY0smcX4G9R7z0XWLK", + "type": "rectangle", + "x": 391, + "y": 318.5, + "width": 849.5, + "height": 97.5, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#f783ac", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "a5V", + "roundness": { + "type": 3 + }, + "seed": 1366218916, + "version": 162, + "versionNonce": 1065698460, + "isDeleted": false, + "boundElements": null, + "updated": 1756974132687, + "link": null, + "locked": false + }, + { + "id": "ZLRaVEUGnkY0TvCSTmXVw", + "type": "rectangle", + "x": 413.25, + "y": 94.5, + "width": 180, + "height": 75.5, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#ffa94d", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "a7", + "roundness": { + "type": 3 + }, + "seed": 1191665180, + "version": 321, + "versionNonce": 1326156956, + "isDeleted": false, + "boundElements": [ + { + "type": "text", + "id": "HtTi1dcJ7e-UGyuIXpAk0" + } + ], + "updated": 1756974367448, + "link": null, + "locked": false + }, + { + "id": "HtTi1dcJ7e-UGyuIXpAk0", + "type": "text", + "x": 420.75011444091797, + "y": 119.75, + "width": 164.99977111816406, + "height": 25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e64980", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "a7V", + "roundness": null, + "seed": 414464932, + "version": 180, + "versionNonce": 531077404, + "isDeleted": false, + "boundElements": null, + "updated": 1756974367448, + "link": null, + "locked": false, + "text": "Feed Forward NN", + "fontSize": 20, + "fontFamily": 8, + "textAlign": "center", + "verticalAlign": "middle", + "containerId": "ZLRaVEUGnkY0TvCSTmXVw", + "originalText": "Feed Forward NN", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "id": "PoIjwif-ulHmWlbvRqDgA", + "type": "rectangle", + "x": 635.25, + "y": 94.5, + "width": 180, + "height": 75.5, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#ffa94d", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "a8", + "roundness": { + "type": 3 + }, + "seed": 1759648164, + "version": 423, + "versionNonce": 168695196, + "isDeleted": false, + "boundElements": [ + { + "type": "text", + "id": "k20eFpCEN_7Xp6evbJfU7" + } + ], + "updated": 1756974367448, + "link": null, + "locked": false + }, + { + "id": "k20eFpCEN_7Xp6evbJfU7", + "type": "text", + "x": 642.750114440918, + "y": 119.75, + "width": 164.99977111816406, + "height": 25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e64980", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "a8V", + "roundness": null, + "seed": 12954916, + "version": 188, + "versionNonce": 1265697308, + "isDeleted": false, + "boundElements": null, + "updated": 1756974367448, + "link": null, + "locked": false, + "text": "Feed Forward NN", + "fontSize": 20, + "fontFamily": 8, + "textAlign": "center", + "verticalAlign": "middle", + "containerId": "PoIjwif-ulHmWlbvRqDgA", + "originalText": "Feed Forward NN", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "id": "x4mTvUvDIBDEPbulzbHcL", + "type": "rectangle", + "x": 1050.25, + "y": 94.5, + "width": 180, + "height": 75.5, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#ffa94d", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "a9", + "roundness": { + "type": 3 + }, + "seed": 1535749660, + "version": 503, + "versionNonce": 717381276, + "isDeleted": false, + "boundElements": [ + { + "type": "text", + "id": "zPnCSR_H5BHUObjMbNeqZ" + } + ], + "updated": 1756974367448, + "link": null, + "locked": false + }, + { + "id": "zPnCSR_H5BHUObjMbNeqZ", + "type": "text", + "x": 1057.750114440918, + "y": 119.75, + "width": 164.99977111816406, + "height": 25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e64980", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "a9V", + "roundness": null, + "seed": 418990756, + "version": 178, + "versionNonce": 1085549340, + "isDeleted": false, + "boundElements": null, + "updated": 1756974367448, + "link": null, + "locked": false, + "text": "Feed Forward NN", + "fontSize": 20, + "fontFamily": 8, + "textAlign": "center", + "verticalAlign": "middle", + "containerId": "x4mTvUvDIBDEPbulzbHcL", + "originalText": "Feed Forward NN", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "id": "x_YA3nwMG-owDQxBe7Tjp", + "type": "line", + "x": 839.5, + "y": 133.5, + "width": 183.5, + "height": 0, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#a5d8ff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aA", + "roundness": { + "type": 2 + }, + "seed": 1593746084, + "version": 384, + "versionNonce": 1069049756, + "isDeleted": false, + "boundElements": null, + "updated": 1756974367448, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 183.5, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": null + }, + { + "id": "3jmnpBPnv80I6tMWIVqpF", + "type": "rectangle", + "x": 420.5, + "y": 331.5, + "width": 165.5, + "height": 69, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e64980", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aB", + "roundness": { + "type": 3 + }, + "seed": 1248010020, + "version": 98, + "versionNonce": 212379428, + "isDeleted": false, + "boundElements": [ + { + "id": "sWlN35snGOcjcdHWRp5Nd", + "type": "arrow" + }, + { + "type": "text", + "id": "wG8fN27Pr9dsVG8c7D6E4" + } + ], + "updated": 1756974213124, + "link": null, + "locked": false + }, + { + "id": "wG8fN27Pr9dsVG8c7D6E4", + "type": "text", + "x": 426.25010681152344, + "y": 353.5, + "width": 153.99978637695312, + "height": 25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e64980", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aBV", + "roundness": null, + "seed": 1664343716, + "version": 16, + "versionNonce": 307238564, + "isDeleted": false, + "boundElements": null, + "updated": 1756974137875, + "link": null, + "locked": false, + "text": "Self Attention", + "fontSize": 20, + "fontFamily": 8, + "textAlign": "center", + "verticalAlign": "middle", + "containerId": "3jmnpBPnv80I6tMWIVqpF", + "originalText": "Self Attention", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "id": "RBEswQ-LRUXR2fgi_ve0s", + "type": "rectangle", + "x": 642.5, + "y": 331.5, + "width": 165.5, + "height": 69, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e64980", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aC", + "roundness": { + "type": 3 + }, + "seed": 169703076, + "version": 181, + "versionNonce": 1683054236, + "isDeleted": false, + "boundElements": [ + { + "id": "nITOk7rv-IXFAnIHJnY98", + "type": "arrow" + }, + { + "id": "sWlN35snGOcjcdHWRp5Nd", + "type": "arrow" + }, + { + "type": "text", + "id": "-3QOujBVRCI1H7W6dniuh" + } + ], + "updated": 1756974139936, + "link": null, + "locked": false + }, + { + "id": "-3QOujBVRCI1H7W6dniuh", + "type": "text", + "x": 648.2501068115234, + "y": 353.5, + "width": 153.99978637695312, + "height": 25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e64980", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aCV", + "roundness": null, + "seed": 1420672420, + "version": 16, + "versionNonce": 1688991140, + "isDeleted": false, + "boundElements": null, + "updated": 1756974143545, + "link": null, + "locked": false, + "text": "Self Attention", + "fontSize": 20, + "fontFamily": 8, + "textAlign": "center", + "verticalAlign": "middle", + "containerId": "RBEswQ-LRUXR2fgi_ve0s", + "originalText": "Self Attention", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "id": "uQUxBLfEIk11ccmn2aL40", + "type": "rectangle", + "x": 1057.5, + "y": 331.5, + "width": 165.5, + "height": 69, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e64980", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aD", + "roundness": { + "type": 3 + }, + "seed": 2003471268, + "version": 275, + "versionNonce": 267825052, + "isDeleted": false, + "boundElements": [ + { + "id": "nITOk7rv-IXFAnIHJnY98", + "type": "arrow" + }, + { + "type": "text", + "id": "zYK_bxlmCi80z7MGnrq5z" + } + ], + "updated": 1756974145493, + "link": null, + "locked": false + }, + { + "id": "zYK_bxlmCi80z7MGnrq5z", + "type": "text", + "x": 1063.2501068115234, + "y": 353.5, + "width": 153.99978637695312, + "height": 25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e64980", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aDV", + "roundness": null, + "seed": 2061360292, + "version": 16, + "versionNonce": 389944484, + "isDeleted": false, + "boundElements": null, + "updated": 1756974148041, + "link": null, + "locked": false, + "text": "Self Attention", + "fontSize": 20, + "fontFamily": 8, + "textAlign": "center", + "verticalAlign": "middle", + "containerId": "uQUxBLfEIk11ccmn2aL40", + "originalText": "Self Attention", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "id": "nITOk7rv-IXFAnIHJnY98", + "type": "arrow", + "x": 815.5, + "y": 365, + "width": 233.5, + "height": 0, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e64980", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aE", + "roundness": { + "type": 2 + }, + "seed": 1555672732, + "version": 43, + "versionNonce": 717616164, + "isDeleted": false, + "boundElements": null, + "updated": 1756974107012, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 233.5, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": { + "elementId": "RBEswQ-LRUXR2fgi_ve0s", + "focus": -0.028985507246376285, + "gap": 7.5 + }, + "endBinding": { + "elementId": "uQUxBLfEIk11ccmn2aL40", + "focus": 0.028985507246377457, + "gap": 8.5 + }, + "startArrowhead": null, + "endArrowhead": "triangle", + "elbowed": false + }, + { + "id": "sWlN35snGOcjcdHWRp5Nd", + "type": "arrow", + "x": 591.5, + "y": 365.5, + "width": 48, + "height": 0, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e64980", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aF", + "roundness": { + "type": 2 + }, + "seed": 1483041948, + "version": 21, + "versionNonce": 301198236, + "isDeleted": false, + "boundElements": null, + "updated": 1756974116073, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 48, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": { + "elementId": "3jmnpBPnv80I6tMWIVqpF", + "focus": -0.014492753623188142, + "gap": 5.5 + }, + "endBinding": { + "elementId": "RBEswQ-LRUXR2fgi_ve0s", + "focus": 0.014492753623188729, + "gap": 3 + }, + "startArrowhead": null, + "endArrowhead": "triangle", + "elbowed": false + }, + { + "id": "4sJ94N1lYR91OeBQ79i2V", + "type": "arrow", + "x": 501, + "y": 469.5, + "width": 0, + "height": 67.00000000000006, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e64980", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aG", + "roundness": { + "type": 2 + }, + "seed": 585768220, + "version": 67, + "versionNonce": 2036556444, + "isDeleted": false, + "boundElements": null, + "updated": 1756974210200, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + -67.00000000000006 + ] + ], + "lastCommittedPoint": null, + "startBinding": { + "elementId": "qvnPO_BO8oXtqwspw9-Gx", + "focus": -0.025495750708215283, + "gap": 1 + }, + "endBinding": { + "elementId": "3jmnpBPnv80I6tMWIVqpF", + "focus": 0.027190332326283918, + "gap": 1.9999999999999432 + }, + "startArrowhead": null, + "endArrowhead": "triangle", + "elbowed": false + }, + { + "id": "TWLyV2I_E4sOPJKsn2KzU", + "type": "arrow", + "x": 503.25, + "y": 330.228342, + "width": 0, + "height": 56.5, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e64980", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aH", + "roundness": { + "type": 2 + }, + "seed": 1394135460, + "version": 73, + "versionNonce": 936890404, + "isDeleted": false, + "boundElements": null, + "updated": 1756974457942, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + -56.5 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "elbowed": false + }, + { + "id": "S-zhjwmfu8yi9KTeRCypb", + "type": "arrow", + "x": 722.1157760000001, + "y": 469.550288, + "width": 0, + "height": 67.00000000000006, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e64980", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aI", + "roundness": { + "type": 2 + }, + "seed": 853810076, + "version": 114, + "versionNonce": 1907489828, + "isDeleted": false, + "boundElements": null, + "updated": 1756974221193, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + -67.00000000000006 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "elbowed": false + }, + { + "id": "fnGm0h9mDfK8dmLyCWdqa", + "type": "arrow", + "x": 724.3657760000001, + "y": 330.27863, + "width": 0, + "height": 56.5, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e64980", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aJ", + "roundness": { + "type": 2 + }, + "seed": 1926692132, + "version": 119, + "versionNonce": 778548252, + "isDeleted": false, + "boundElements": null, + "updated": 1756974457942, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + -56.5 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "elbowed": false + }, + { + "id": "kvIRlZGS_dM1mixZXndK5", + "type": "arrow", + "x": 1139.1248320000002, + "y": 469.550288, + "width": 0, + "height": 67.00000000000006, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e64980", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aK", + "roundness": { + "type": 2 + }, + "seed": 867017508, + "version": 220, + "versionNonce": 1802812836, + "isDeleted": false, + "boundElements": null, + "updated": 1756974226250, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + -67.00000000000006 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "elbowed": false + }, + { + "id": "j019u_6nu4js_5MhJ0yR5", + "type": "arrow", + "x": 1141.3748320000002, + "y": 330.27863, + "width": 0, + "height": 56.5, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e64980", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aL", + "roundness": { + "type": 2 + }, + "seed": 1623439524, + "version": 225, + "versionNonce": 338832292, + "isDeleted": false, + "boundElements": null, + "updated": 1756974457942, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + -56.5 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "elbowed": false + }, + { + "id": "bTjw5oQwxiUK82pb2b6E5", + "type": "rectangle", + "x": 415, + "y": -47.21975299999997, + "width": 176.5, + "height": 49.5, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#d0bfff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aM", + "roundness": { + "type": 3 + }, + "seed": 1690089636, + "version": 573, + "versionNonce": 1044174116, + "isDeleted": false, + "boundElements": [ + { + "type": "text", + "id": "oF5QUT_oGIeRYDowPq5m6" + } + ], + "updated": 1756974485123, + "link": null, + "locked": false + }, + { + "id": "oF5QUT_oGIeRYDowPq5m6", + "type": "text", + "x": 492.25001525878906, + "y": -34.96975299999997, + "width": 21.999969482421875, + "height": 25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#9775fa", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aMV", + "roundness": null, + "seed": 2125083292, + "version": 4, + "versionNonce": 987628700, + "isDeleted": false, + "boundElements": null, + "updated": 1756974489121, + "link": null, + "locked": false, + "text": "R1", + "fontSize": 20, + "fontFamily": 8, + "textAlign": "center", + "verticalAlign": "middle", + "containerId": "bTjw5oQwxiUK82pb2b6E5", + "originalText": "R1", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "id": "T3NC0qcRsIvMNPovl9T-1", + "type": "rectangle", + "x": 637, + "y": -47.21975299999997, + "width": 176.5, + "height": 49.5, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#d0bfff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aN", + "roundness": { + "type": 3 + }, + "seed": 359007268, + "version": 840, + "versionNonce": 2110499364, + "isDeleted": false, + "boundElements": [ + { + "type": "text", + "id": "2FwbUBYaGc6NT89hEbAcZ" + } + ], + "updated": 1756974490759, + "link": null, + "locked": false + }, + { + "id": "2FwbUBYaGc6NT89hEbAcZ", + "type": "text", + "x": 714.2500152587891, + "y": -34.96975299999997, + "width": 21.999969482421875, + "height": 25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#9775fa", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aNV", + "roundness": null, + "seed": 228756892, + "version": 4, + "versionNonce": 2035343260, + "isDeleted": false, + "boundElements": null, + "updated": 1756974491612, + "link": null, + "locked": false, + "text": "R2", + "fontSize": 20, + "fontFamily": 8, + "textAlign": "center", + "verticalAlign": "middle", + "containerId": "T3NC0qcRsIvMNPovl9T-1", + "originalText": "R2", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "id": "v_SajmUIB69yvNX7pJYhF", + "type": "rectangle", + "x": 1052, + "y": -47.21975299999997, + "width": 176.5, + "height": 49.5, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#d0bfff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aO", + "roundness": { + "type": 3 + }, + "seed": 165273508, + "version": 960, + "versionNonce": 1802944292, + "isDeleted": false, + "boundElements": [ + { + "type": "text", + "id": "ighjroAq4YQxs1mOhrLL_" + } + ], + "updated": 1756974493768, + "link": null, + "locked": false + }, + { + "id": "ighjroAq4YQxs1mOhrLL_", + "type": "text", + "x": 1129.250015258789, + "y": -34.96975299999997, + "width": 21.999969482421875, + "height": 25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#9775fa", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aOV", + "roundness": null, + "seed": 190490780, + "version": 4, + "versionNonce": 1755544220, + "isDeleted": false, + "boundElements": null, + "updated": 1756974494786, + "link": null, + "locked": false, + "text": "RN", + "fontSize": 20, + "fontFamily": 8, + "textAlign": "center", + "verticalAlign": "middle", + "containerId": "v_SajmUIB69yvNX7pJYhF", + "originalText": "RN", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "id": "vk98lbc9sIkJGg49NbFqr", + "type": "line", + "x": 830, + "y": -25.21975299999997, + "width": 202, + "height": 0, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#d0bfff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aP", + "roundness": { + "type": 2 + }, + "seed": 172378916, + "version": 509, + "versionNonce": 1551001884, + "isDeleted": false, + "boundElements": null, + "updated": 1756974391811, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 202, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": null + }, + { + "id": "FvpkY6ESZR40PKAm8bxbH", + "type": "arrow", + "x": 503.25, + "y": 92.12070741986125, + "width": 0, + "height": 88.4055971074039, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e64980", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aQ", + "roundness": { + "type": 2 + }, + "seed": 174070172, + "version": 352, + "versionNonce": 1123558940, + "isDeleted": false, + "boundElements": null, + "updated": 1756974367448, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + -88.4055971074039 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "elbowed": false + }, + { + "id": "qrr1jVTlWpGDSNpAPXYlZ", + "type": "arrow", + "x": 724.3657760000001, + "y": 92.19939309538938, + "width": 0, + "height": 88.4055971074039, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e64980", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aR", + "roundness": { + "type": 2 + }, + "seed": 932595740, + "version": 398, + "versionNonce": 1841627804, + "isDeleted": false, + "boundElements": null, + "updated": 1756974367448, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + -88.4055971074039 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "elbowed": false + }, + { + "id": "-pkarzxxAdB6nJFkN4cNr", + "type": "arrow", + "x": 1141.3748320000002, + "y": 92.19939309538938, + "width": 0, + "height": 88.4055971074039, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e64980", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aS", + "roundness": { + "type": 2 + }, + "seed": 1449091364, + "version": 504, + "versionNonce": 1405243164, + "isDeleted": false, + "boundElements": null, + "updated": 1756974367448, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + -88.4055971074039 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "elbowed": false + }, + { + "id": "qvnPO_BO8oXtqwspw9-Gx", + "type": "rectangle", + "x": 415, + "y": 219.78024700000003, + "width": 176.5, + "height": 49.5, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#9775fa", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aT", + "roundness": { + "type": 3 + }, + "seed": 308494748, + "version": 703, + "versionNonce": 1504161820, + "isDeleted": false, + "boundElements": [ + { + "type": "text", + "id": "mX5X9W9xkcQ8BbFMz_vsq" + } + ], + "updated": 1756974498349, + "link": null, + "locked": false + }, + { + "id": "mX5X9W9xkcQ8BbFMz_vsq", + "type": "text", + "x": 492.25001525878906, + "y": 232.03024700000003, + "width": 21.999969482421875, + "height": 25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#9775fa", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aTV", + "roundness": null, + "seed": 1426986020, + "version": 4, + "versionNonce": 1579213348, + "isDeleted": false, + "boundElements": null, + "updated": 1756974499630, + "link": null, + "locked": false, + "text": "Z1", + "fontSize": 20, + "fontFamily": 8, + "textAlign": "center", + "verticalAlign": "middle", + "containerId": "qvnPO_BO8oXtqwspw9-Gx", + "originalText": "Z1", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "id": "mju6Tto5F4RM4UCKpXtrs", + "type": "rectangle", + "x": 637, + "y": 219.78024700000003, + "width": 176.5, + "height": 49.5, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#9775fa", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aU", + "roundness": { + "type": 3 + }, + "seed": 99458844, + "version": 970, + "versionNonce": 1120135964, + "isDeleted": false, + "boundElements": [ + { + "type": "text", + "id": "4K5valzn0LKg5Cvdtj_AB" + } + ], + "updated": 1756974501047, + "link": null, + "locked": false + }, + { + "id": "4K5valzn0LKg5Cvdtj_AB", + "type": "text", + "x": 714.2500152587891, + "y": 232.03024700000003, + "width": 21.999969482421875, + "height": 25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#9775fa", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aUV", + "roundness": null, + "seed": 1818339620, + "version": 4, + "versionNonce": 142735140, + "isDeleted": false, + "boundElements": null, + "updated": 1756974502263, + "link": null, + "locked": false, + "text": "Z2", + "fontSize": 20, + "fontFamily": 8, + "textAlign": "center", + "verticalAlign": "middle", + "containerId": "mju6Tto5F4RM4UCKpXtrs", + "originalText": "Z2", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "id": "XuOOzkT2R_FitW-sQ8lFa", + "type": "rectangle", + "x": 1052, + "y": 219.78024700000003, + "width": 176.5, + "height": 49.5, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#9775fa", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aV", + "roundness": { + "type": 3 + }, + "seed": 210696996, + "version": 1094, + "versionNonce": 364725156, + "isDeleted": false, + "boundElements": [ + { + "type": "text", + "id": "z3J-L_P1vMKbzOrYzEmN8" + } + ], + "updated": 1756974510039, + "link": null, + "locked": false + }, + { + "id": "z3J-L_P1vMKbzOrYzEmN8", + "type": "text", + "x": 1129.250015258789, + "y": 232.03024700000003, + "width": 21.999969482421875, + "height": 25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#9775fa", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aVG", + "roundness": null, + "seed": 1692867612, + "version": 4, + "versionNonce": 1687825948, + "isDeleted": false, + "boundElements": null, + "updated": 1756974511355, + "link": null, + "locked": false, + "text": "ZN", + "fontSize": 20, + "fontFamily": 8, + "textAlign": "center", + "verticalAlign": "middle", + "containerId": "XuOOzkT2R_FitW-sQ8lFa", + "originalText": "ZN", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "id": "bDC6SqgoxEyH55UyhffID", + "type": "line", + "x": 830, + "y": 241.78024700000003, + "width": 202, + "height": 0, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#9775fa", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aW", + "roundness": { + "type": 2 + }, + "seed": 381266724, + "version": 639, + "versionNonce": 618063516, + "isDeleted": false, + "boundElements": null, + "updated": 1756974434975, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 202, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": null + }, + { + "id": "m_8hbxJAgtwziyaCZXtvZ", + "type": "arrow", + "x": 503.25, + "y": 219.4403430355035, + "width": 0, + "height": 49.29688876559714, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e64980", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aX", + "roundness": { + "type": 2 + }, + "seed": 1876867364, + "version": 141, + "versionNonce": 1564579740, + "isDeleted": false, + "boundElements": null, + "updated": 1756974469873, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + -49.29688876559714 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "elbowed": false + }, + { + "id": "QLNhoKV5qULjOIbRWCpLM", + "type": "arrow", + "x": 724.3657760000001, + "y": 219.48421988403882, + "width": 0, + "height": 49.29688876559714, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e64980", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aY", + "roundness": { + "type": 2 + }, + "seed": 1567880356, + "version": 187, + "versionNonce": 1123696676, + "isDeleted": false, + "boundElements": null, + "updated": 1756974469873, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + -49.29688876559714 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "elbowed": false + }, + { + "id": "E42erKiGCWdR8_tNB3U3V", + "type": "arrow", + "x": 1141.3748320000002, + "y": 219.48421988403882, + "width": 0, + "height": 49.29688876559714, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e64980", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aZ", + "roundness": { + "type": 2 + }, + "seed": 1809170468, + "version": 293, + "versionNonce": 820266012, + "isDeleted": false, + "boundElements": null, + "updated": 1756974469873, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + -49.29688876559714 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "elbowed": false + } + ], + "appState": { + "gridSize": 20, + "gridStep": 5, + "gridModeEnabled": false, + "viewBackgroundColor": "#ffffff" + }, + "files": {} +} \ No newline at end of file diff --git a/Chapters/12-Transformers/Images/Excalidraw/transformer-high-level.excalidraw.json b/Chapters/12-Transformers/Images/Excalidraw/transformer-high-level.excalidraw.json new file mode 100644 index 0000000..bac251b --- /dev/null +++ b/Chapters/12-Transformers/Images/Excalidraw/transformer-high-level.excalidraw.json @@ -0,0 +1,1292 @@ +{ + "type": "excalidraw", + "version": 2, + "source": "https://marketplace.visualstudio.com/items?itemName=pomdtr.excalidraw-editor", + "elements": [ + { + "id": "qks0kFfyC68OY6cMZd6P_", + "type": "rectangle", + "x": 432, + "y": 128.99999999999994, + "width": 598.5000000000001, + "height": 314.50000000000006, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aq", + "roundness": { + "type": 3 + }, + "seed": 1684020354, + "version": 392, + "versionNonce": 311577905, + "isDeleted": false, + "boundElements": [ + { + "id": "lt4HrBhjlnadLJxkbipCy", + "type": "arrow" + } + ], + "updated": 1756972060356, + "link": null, + "locked": false + }, + { + "id": "Y3tqkOGPwoDNYpZGYgVLa", + "type": "rectangle", + "x": 453, + "y": 145.125, + "width": 216.5, + "height": 37.25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#b2f2bb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "az", + "roundness": { + "type": 3 + }, + "seed": 850175454, + "version": 531, + "versionNonce": 806703234, + "isDeleted": false, + "boundElements": [ + { + "id": "WojlgWROz7R4TQ92aLzS-", + "type": "arrow" + }, + { + "id": "CcLYgRdqiPMRm53uXlGEe", + "type": "arrow" + } + ], + "updated": 1756971712324, + "link": null, + "locked": false + }, + { + "id": "FhNS-O1GijMBdVynOjCaD", + "type": "rectangle", + "x": 793, + "y": 147.125, + "width": 216.5, + "height": 37.25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#ffc9c9", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "b02", + "roundness": { + "type": 3 + }, + "seed": 50557086, + "version": 760, + "versionNonce": 1192062001, + "isDeleted": false, + "boundElements": [ + { + "id": "CcLYgRdqiPMRm53uXlGEe", + "type": "arrow" + }, + { + "id": "MsALcikZJRpuD_FzPE0Ec", + "type": "arrow" + }, + { + "id": "FTyJ5gZy1dwVkj-dHctLF", + "type": "arrow" + } + ], + "updated": 1756971987940, + "link": null, + "locked": false + }, + { + "id": "svCHgKAR1hgOhcQ2JVqnF", + "type": "rectangle", + "x": 371, + "y": 476.5, + "width": 322.5, + "height": 60.5, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#99e9f2", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "b03", + "roundness": { + "type": 3 + }, + "seed": 1839106910, + "version": 179, + "versionNonce": 1835836223, + "isDeleted": false, + "boundElements": [ + { + "type": "text", + "id": "XjbZ2XJCBia1DnIGRU6cC" + }, + { + "id": "WojlgWROz7R4TQ92aLzS-", + "type": "arrow" + }, + { + "id": "3PWmJix_8Q6bO8UqWsgzv", + "type": "arrow" + } + ], + "updated": 1756971957816, + "link": null, + "locked": false + }, + { + "id": "XjbZ2XJCBia1DnIGRU6cC", + "type": "text", + "x": 449.75011444091797, + "y": 494.25, + "width": 164.99977111816406, + "height": 25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#ffec99", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "b03V", + "roundness": null, + "seed": 1919489154, + "version": 39, + "versionNonce": 2100433602, + "isDeleted": false, + "boundElements": [], + "updated": 1756971428921, + "link": null, + "locked": false, + "text": "Sono molto alto", + "fontSize": 20, + "fontFamily": 8, + "textAlign": "center", + "verticalAlign": "middle", + "containerId": "svCHgKAR1hgOhcQ2JVqnF", + "originalText": "Sono molto alto", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "id": "dNW_G75JMy2DV1tXZouKq", + "type": "rectangle", + "x": 778, + "y": 30.125, + "width": 322.5, + "height": 60.5, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#ffec99", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "b04", + "roundness": { + "type": 3 + }, + "seed": 80631554, + "version": 282, + "versionNonce": 630235743, + "isDeleted": false, + "boundElements": [ + { + "type": "text", + "id": "v4Yzc4KJZezs6eZdUx5ud" + }, + { + "id": "MsALcikZJRpuD_FzPE0Ec", + "type": "arrow" + } + ], + "updated": 1756972086244, + "link": null, + "locked": false + }, + { + "id": "v4Yzc4KJZezs6eZdUx5ud", + "type": "text", + "x": 867.7500991821289, + "y": 47.875, + "width": 142.9998016357422, + "height": 25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#ffec99", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "b05", + "roundness": null, + "seed": 63648258, + "version": 36, + "versionNonce": 959118975, + "isDeleted": false, + "boundElements": [], + "updated": 1756972086244, + "link": null, + "locked": false, + "text": "I'm very tall", + "fontSize": 20, + "fontFamily": 8, + "textAlign": "center", + "verticalAlign": "middle", + "containerId": "dNW_G75JMy2DV1tXZouKq", + "originalText": "I'm very tall", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "id": "WojlgWROz7R4TQ92aLzS-", + "type": "arrow", + "x": 560.6729092706995, + "y": 388.3125, + "width": 0, + "height": 205.0903822278525, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#ffec99", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "b06", + "roundness": { + "type": 2 + }, + "seed": 503111106, + "version": 366, + "versionNonce": 1116642929, + "isDeleted": false, + "boundElements": [], + "updated": 1756971948357, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + -205.0903822278525 + ] + ], + "lastCommittedPoint": null, + "startBinding": { + "elementId": "XJ45iAj9hZYYHBBzyfbWx", + "focus": 0, + "gap": 8.4375 + }, + "endBinding": { + "elementId": "Y3tqkOGPwoDNYpZGYgVLa", + "focus": 0.005331092187533408, + "gap": 1 + }, + "startArrowhead": null, + "endArrowhead": "triangle", + "elbowed": false + }, + { + "id": "3WKvAEkmIviNstZ7kyy2b", + "type": "rectangle", + "x": 449.5, + "y": 387.25, + "width": 216.5, + "height": 35, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#b2f2bb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "b07", + "roundness": { + "type": 3 + }, + "seed": 1179987230, + "version": 368, + "versionNonce": 2132680575, + "isDeleted": false, + "boundElements": [ + { + "type": "text", + "id": "_WIy_VWxm2zUNEW9Z1h-T" + }, + { + "id": "3PWmJix_8Q6bO8UqWsgzv", + "type": "arrow" + } + ], + "updated": 1756971957817, + "link": null, + "locked": false + }, + { + "id": "_WIy_VWxm2zUNEW9Z1h-T", + "type": "text", + "x": 519.2500534057617, + "y": 392.25, + "width": 76.99989318847656, + "height": 25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "b08", + "roundness": null, + "seed": 680505282, + "version": 22, + "versionNonce": 1406241566, + "isDeleted": false, + "boundElements": [], + "updated": 1756971468891, + "link": null, + "locked": false, + "text": "Encoder", + "fontSize": 20, + "fontFamily": 8, + "textAlign": "center", + "verticalAlign": "middle", + "containerId": "3WKvAEkmIviNstZ7kyy2b", + "originalText": "Encoder", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "id": "XJ45iAj9hZYYHBBzyfbWx", + "type": "rectangle", + "x": 449.5, + "y": 342.625, + "width": 216.5, + "height": 37.25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#b2f2bb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "b09", + "roundness": { + "type": 3 + }, + "seed": 1135466846, + "version": 387, + "versionNonce": 1432033489, + "isDeleted": false, + "boundElements": [ + { + "id": "WojlgWROz7R4TQ92aLzS-", + "type": "arrow" + } + ], + "updated": 1756971947784, + "link": null, + "locked": false + }, + { + "id": "B1_Gl5jIVNIVEm4cqQjEO", + "type": "rectangle", + "x": 449.5, + "y": 298, + "width": 216.5, + "height": 37.25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#b2f2bb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "b0A", + "roundness": { + "type": 3 + }, + "seed": 564592030, + "version": 419, + "versionNonce": 1575909214, + "isDeleted": false, + "boundElements": [ + { + "id": "WojlgWROz7R4TQ92aLzS-", + "type": "arrow" + } + ], + "updated": 1756971468891, + "link": null, + "locked": false + }, + { + "id": "M0Sabot8-Z9iGzHo1b0Hp", + "type": "rectangle", + "x": 453, + "y": 253.375, + "width": 216.5, + "height": 37.25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#b2f2bb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "b0B", + "roundness": { + "type": 3 + }, + "seed": 469493086, + "version": 406, + "versionNonce": 400405890, + "isDeleted": false, + "boundElements": [], + "updated": 1756971468891, + "link": null, + "locked": false + }, + { + "id": "TWt9CwZmm4N_Bb-2IZ6vx", + "type": "rectangle", + "x": 453, + "y": 208.75, + "width": 216.5, + "height": 37.25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#b2f2bb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "b0C", + "roundness": { + "type": 3 + }, + "seed": 1548978590, + "version": 428, + "versionNonce": 639884190, + "isDeleted": false, + "boundElements": [], + "updated": 1756971468891, + "link": null, + "locked": false + }, + { + "id": "CcLYgRdqiPMRm53uXlGEe", + "type": "arrow", + "x": 670.4842460920306, + "y": 163.56982573606547, + "width": 119.0823985958674, + "height": 3.5698257360654395, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#ffec99", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "b0E", + "roundness": { + "type": 2 + }, + "seed": 411246466, + "version": 90, + "versionNonce": 136988610, + "isDeleted": false, + "boundElements": [], + "updated": 1756971712325, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 119.0823985958674, + -3.5698257360654395 + ] + ], + "lastCommittedPoint": null, + "startBinding": { + "elementId": "Y3tqkOGPwoDNYpZGYgVLa", + "focus": -0.2013422818791942, + "gap": 1 + }, + "endBinding": { + "elementId": "FhNS-O1GijMBdVynOjCaD", + "focus": 0.30872483221476527, + "gap": 3.5 + }, + "startArrowhead": null, + "endArrowhead": "triangle", + "elbowed": false + }, + { + "id": "lt4HrBhjlnadLJxkbipCy", + "type": "arrow", + "x": 699.5, + "y": 160.5, + "width": 89.9466241174614, + "height": 244.60112529417955, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#ffec99", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "b0I", + "roundness": { + "type": 2 + }, + "seed": 1821635330, + "version": 296, + "versionNonce": 766953886, + "isDeleted": false, + "boundElements": [], + "updated": 1756971652850, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 220.5 + ], + [ + 89.9466241174614, + 244.60112529417955 + ] + ], + "lastCommittedPoint": null, + "startBinding": { + "elementId": "qks0kFfyC68OY6cMZd6P_", + "focus": 0, + "gap": 31.500000000000057 + }, + "endBinding": { + "elementId": "pe1XqPJqxlQiirlcP1WBT", + "focus": -0.5930218526287061, + "gap": 1 + }, + "startArrowhead": null, + "endArrowhead": "triangle", + "elbowed": false + }, + { + "id": "BEYZYL1aLCLJxYwuzEbje", + "type": "arrow", + "x": 693.5, + "y": 363.5, + "width": 88.20191330397734, + "height": 0, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#ffec99", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "b0J", + "roundness": null, + "seed": 1127104625, + "version": 280, + "versionNonce": 918175551, + "isDeleted": false, + "boundElements": null, + "updated": 1756971885498, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 45.5, + 0 + ], + [ + 88.20191330397734, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": { + "elementId": "r5lpc_O1Itbdt60WZylIP", + "focus": -0.12080536912751713, + "gap": 8 + }, + "startArrowhead": null, + "endArrowhead": "triangle", + "elbowed": false + }, + { + "id": "PGF7Gn3DM_BBqX7B0Io_W", + "type": "arrow", + "x": 693.5, + "y": 316.5, + "width": 88.20191330397734, + "height": 0, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#ffec99", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "b0K", + "roundness": null, + "seed": 167605745, + "version": 356, + "versionNonce": 926014879, + "isDeleted": false, + "boundElements": null, + "updated": 1756971898461, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 45.5, + 0 + ], + [ + 88.20191330397734, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "elbowed": false + }, + { + "id": "WLrR3J1Bd9ankMnwNwLEf", + "type": "arrow", + "x": 693.5, + "y": 272, + "width": 88.20191330397734, + "height": 0, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#ffec99", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "b0L", + "roundness": null, + "seed": 725429567, + "version": 381, + "versionNonce": 342264369, + "isDeleted": false, + "boundElements": null, + "updated": 1756971912107, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 45.5, + 0 + ], + [ + 88.20191330397734, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "elbowed": false + }, + { + "id": "IRBUK1mWePuv6ADRZvWj5", + "type": "arrow", + "x": 696, + "y": 228, + "width": 88.20191330397734, + "height": 0, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#ffec99", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "b0M", + "roundness": null, + "seed": 296450590, + "version": 459, + "versionNonce": 2986783, + "isDeleted": false, + "boundElements": [], + "updated": 1756971917092, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 45.5, + 0 + ], + [ + 88.20191330397734, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "elbowed": false + }, + { + "id": "3PWmJix_8Q6bO8UqWsgzv", + "type": "arrow", + "x": 558.75, + "y": 471.8125, + "width": 0, + "height": 49, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "b0N", + "roundness": { + "type": 2 + }, + "seed": 934707519, + "version": 44, + "versionNonce": 1496434015, + "isDeleted": false, + "boundElements": null, + "updated": 1756971964826, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + -49 + ] + ], + "lastCommittedPoint": null, + "startBinding": { + "elementId": "svCHgKAR1hgOhcQ2JVqnF", + "focus": 0.16434108527131783, + "gap": 4.6875 + }, + "endBinding": { + "elementId": "3WKvAEkmIviNstZ7kyy2b", + "focus": -0.009237875288683634, + "gap": 1 + }, + "startArrowhead": null, + "endArrowhead": "triangle", + "elbowed": false + }, + { + "id": "MsALcikZJRpuD_FzPE0Ec", + "type": "arrow", + "x": 899.2500000000002, + "y": 146.4858737337955, + "width": 1.1368683772161603e-13, + "height": 55.610715303298534, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "b0O", + "roundness": { + "type": 2 + }, + "seed": 1380342257, + "version": 74, + "versionNonce": 2103760543, + "isDeleted": false, + "boundElements": null, + "updated": 1756972086244, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + -1.1368683772161603e-13, + -55.610715303298534 + ] + ], + "lastCommittedPoint": null, + "startBinding": { + "elementId": "FhNS-O1GijMBdVynOjCaD", + "focus": -0.01847575057736726, + "gap": 3.3125 + }, + "endBinding": { + "elementId": "dNW_G75JMy2DV1tXZouKq", + "focus": 0.2480620155038739, + "gap": 1.3125 + }, + "startArrowhead": null, + "endArrowhead": "triangle", + "elbowed": false + }, + { + "id": "FTyJ5gZy1dwVkj-dHctLF", + "type": "arrow", + "x": 900.75, + "y": 389.8125, + "width": 0, + "height": 207, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "b0V", + "roundness": { + "type": 2 + }, + "seed": 1481141809, + "version": 56, + "versionNonce": 806864881, + "isDeleted": false, + "boundElements": null, + "updated": 1756972007338, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + -207 + ] + ], + "lastCommittedPoint": null, + "startBinding": { + "elementId": "r5lpc_O1Itbdt60WZylIP", + "focus": 0, + "gap": 9.9375 + }, + "endBinding": { + "elementId": "FhNS-O1GijMBdVynOjCaD", + "focus": 0.004618937644341804, + "gap": 1.5625 + }, + "startArrowhead": null, + "endArrowhead": "triangle", + "elbowed": false + }, + { + "id": "pe1XqPJqxlQiirlcP1WBT", + "type": "rectangle", + "x": 789.5, + "y": 387.25, + "width": 216.5, + "height": 37.25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#ffc9c9", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "b0W", + "roundness": { + "type": 3 + }, + "seed": 1701943774, + "version": 639, + "versionNonce": 2062601201, + "isDeleted": false, + "boundElements": [ + { + "type": "text", + "id": "x5N2jQsZ9-BwKSn2CoHVI" + }, + { + "id": "lt4HrBhjlnadLJxkbipCy", + "type": "arrow" + } + ], + "updated": 1756972002259, + "link": null, + "locked": false + }, + { + "id": "x5N2jQsZ9-BwKSn2CoHVI", + "type": "text", + "x": 859.2500534057617, + "y": 393.375, + "width": 76.99989318847656, + "height": 25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#ffc9c9", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "b0X", + "roundness": null, + "seed": 408652830, + "version": 13, + "versionNonce": 38201247, + "isDeleted": false, + "boundElements": [], + "updated": 1756972002259, + "link": null, + "locked": false, + "text": "Decoder", + "fontSize": 20, + "fontFamily": 8, + "textAlign": "center", + "verticalAlign": "middle", + "containerId": "pe1XqPJqxlQiirlcP1WBT", + "originalText": "Decoder", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "id": "r5lpc_O1Itbdt60WZylIP", + "type": "rectangle", + "x": 789.5, + "y": 342.625, + "width": 216.5, + "height": 37.25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#ffc9c9", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "b0Y", + "roundness": { + "type": 3 + }, + "seed": 1176415774, + "version": 660, + "versionNonce": 1471423953, + "isDeleted": false, + "boundElements": [ + { + "id": "FTyJ5gZy1dwVkj-dHctLF", + "type": "arrow" + } + ], + "updated": 1756972002259, + "link": null, + "locked": false + }, + { + "id": "UtH80dFh6PeThMxTglJ_0", + "type": "rectangle", + "x": 789.5, + "y": 298, + "width": 216.5, + "height": 37.25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#ffc9c9", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "b0Z", + "roundness": { + "type": 3 + }, + "seed": 1889483358, + "version": 687, + "versionNonce": 1231461311, + "isDeleted": false, + "boundElements": [], + "updated": 1756972002259, + "link": null, + "locked": false + }, + { + "id": "c157ucNOlGnUEDjkdsRSF", + "type": "rectangle", + "x": 793, + "y": 253.375, + "width": 216.5, + "height": 37.25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#ffc9c9", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "b0a", + "roundness": { + "type": 3 + }, + "seed": 180584606, + "version": 675, + "versionNonce": 1521566641, + "isDeleted": false, + "boundElements": [], + "updated": 1756972002259, + "link": null, + "locked": false + }, + { + "id": "ZbRQkR2qOkbpqKUcauAuX", + "type": "rectangle", + "x": 793, + "y": 208.75, + "width": 216.5, + "height": 37.25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#ffc9c9", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "b0b", + "roundness": { + "type": 3 + }, + "seed": 1034683102, + "version": 697, + "versionNonce": 1455082463, + "isDeleted": false, + "boundElements": [], + "updated": 1756972002259, + "link": null, + "locked": false + }, + { + "id": "iSjQVuu5ywdRyBWZjnc7w", + "type": "text", + "x": 629.75, + "y": 509.3125, + "width": 52.80010986328125, + "height": 20, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "b0c", + "roundness": null, + "seed": 1823510495, + "version": 66, + "versionNonce": 1359014943, + "isDeleted": false, + "boundElements": null, + "updated": 1756972029220, + "link": null, + "locked": false, + "text": "[0, N]", + "fontSize": 16, + "fontFamily": 8, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "[0, N]", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "id": "Hy2r5uN9__WKZsY91p6ZL", + "type": "text", + "x": 1036.25, + "y": 66.3125, + "width": 52.80010986328125, + "height": 20, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "b0d", + "roundness": null, + "seed": 1025198353, + "version": 102, + "versionNonce": 787272529, + "isDeleted": false, + "boundElements": null, + "updated": 1756972090192, + "link": null, + "locked": false, + "text": "[0, M]", + "fontSize": 16, + "fontFamily": 8, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "[0, M]", + "autoResize": true, + "lineHeight": 1.25 + } + ], + "appState": { + "gridSize": 20, + "gridStep": 5, + "gridModeEnabled": false, + "viewBackgroundColor": "#ffffff" + }, + "files": {} +} \ No newline at end of file diff --git a/Chapters/12-Transformers/Images/PNGs/encoder.png b/Chapters/12-Transformers/Images/PNGs/encoder.png new file mode 100644 index 0000000..74e2fb6 Binary files /dev/null and b/Chapters/12-Transformers/Images/PNGs/encoder.png differ diff --git a/Chapters/12-Transformers/Images/PNGs/transformer-high-level.png b/Chapters/12-Transformers/Images/PNGs/transformer-high-level.png new file mode 100644 index 0000000..cd1d4f4 Binary files /dev/null and b/Chapters/12-Transformers/Images/PNGs/transformer-high-level.png differ