# Transformer

The 3 most important keys to understanding transformer are:&#x20;

1. Position Encoding: allows parallel training&#x20;

&#x20;                                       ![](https://1026108543-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfMiBkUCO0YX35Q3Joj8n%2Fuploads%2FQRNa6fVlJWjRxTfWnlWT%2Fimage.png?alt=media\&token=7902ad1a-e26c-435e-8c1b-1512dd8bb4b2)

&#x20; 2\.   Encoder Decoder mechanism

&#x20;                                      ![](https://1026108543-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfMiBkUCO0YX35Q3Joj8n%2Fuploads%2FpdEqyKZiktmCIRce2YIm%2Fimage.png?alt=media\&token=ac36f94d-729a-4748-8c21-fc1fa64c667d)

&#x20; 3\.   (Multi-head) (Self) Attention Mechanism

&#x20;                              ![](https://1026108543-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfMiBkUCO0YX35Q3Joj8n%2Fuploads%2FrhXLyW1YH819cNDHNdUU%2Fimage.png?alt=media\&token=595a1a17-a446-4fa5-b59f-7f9202e871d4)

Make sure to watch the following three videos "in order" so that you wouldn't get overwhelmed immediately.

{% embed url="<https://www.youtube.com/watch?v=SZorAJ4I-sA>" %}

{% embed url="<https://www.youtube.com/watch?v=FWFA4DGuzSc>" %}

{% embed url="<https://www.youtube.com/watch?v=TQQlZhbC5ps>" %}

* Note: to enable the network "learn" while doing attention, input is projected to a lower dimension using $$H$$ different linear projections (heads).$$H$$ different attentions will be learned from those H heads and added together to yield a final result.
