Computer Vision
  • Introduction
    • Neural network basics
    • MLE and Cross Entropy
    • Convolution basics
    • Neural Network Categories
  • 2D Backbones
    • ResNet
    • Transformer
      • Recurrent Neural Network
      • Vision Transformer
      • SwinTransformer
  • Methods for Object Detection
  • Object Detection
    • The R-CNN family
    • ROI pool & ROI align
    • FCOS
    • Object Detection in Detectron2
  • Segmentation
    • Fully Convolutional Network
    • Unet: image segmentation
  • Video Understanding
    • I3D: video understanding
    • Slowfast: video recognition
    • ActionFormer: temporal action localization
  • Generative models
    • Autoregressive model
    • Variational Auto-Encoder
    • Generative Adversarial Network
    • Diffusion Models
    • 3D Face Reconstruction
Powered by GitBook
On this page
  1. Introduction

MLE and Cross Entropy

PreviousNeural network basicsNextConvolution basics

Last updated 1 year ago

Because maximizing "log likelihood" is equals as minimizing "negative log likelihood", negative log likelihood can be view as a loss function.

  • An important distinction between Negative Log Likelihood Loss (NLLLoss) and Cross Entropy Loss (CE) is that CE implicitly applies a softmax activation followed by a log transformation but NLLLoss does not.

Cross-Entropy, Negative Log-Likelihood, and All That JazzMedium
Logo
Log likelihood for binary classification. Note that we can view this as only caring about the likelihood of the true class.
Log Likelihood for multiclass classification.
given two probability distributions p and q, their cross-entropy is defined as this term