The process of producing a new value of some type from a value of a different type is called a conversion.

Conversions do not change the value or type being converted. Instead, a new value with the desired type is created as a result of the conversion

Functions can be overloaded so long as each overloaded function can be differentiated by the compiler. If an overloaded function can not be differentiated, a compile error will result.

Self-supervised Learning

Self-supervised learning is a form of unsupervised learning. In the example below, we split the input into two parts: and . serves as the input to the model, while contains the remaining information, which the model uses to learn the label information.

Network as Generator

Compared to a traditional neural network, a generator network takes a simple distribution as input and outputs a distribution.

Softmax vs A-softmax Loss

Softmax loss is typically good at optimizing the inter-class difference (separating different classes), but not good at reducing the intra-class variation (making features of the same class compact). Additive Margin Softmax is a novel and more interpretable way to import the angular margin into the softmax loss.

If we intend to run our program many times, it's necessary for us to do test.

介绍各种各样的自注意力机制

mask_fill 是 PyTorch 中的一个函数，用于根据给定的条件（掩码）对张量中的元素进行填充。掩码是一个布尔值张量，指示哪些元素需要被填充。

简结的矩阵运算方法

本文记录阅读Conformer的笔记，以此来达成李宏毅2022深度学习HW4的strong line