List: DL Concept | Curated by Simon Kwan

Feb 24, 2025
170 stories
DL Concept
In
TDS Archive
by
Reza Bagheri
A Visual Understanding of the Softmax FunctionThe math and intuition behind the softmax function and its application in neural networks and softmax regression
Nov 3, 2024
2
Nov 3, 2024
2
Vitality Learning
Harnessing Singular Value Decomposition (SVD) for Efficient Neural Network Weight PruningAs the field of deep learning continues to evolve, one challenge remains constant: the trade-off between model complexity and computational…
Aug 10, 2024
Aug 10, 2024
Sergey K.
ReAct- Rectified Activations for OOD data detectionAn overview of post-hoc method presented in 2021 paper for out-of-distribution detection task.
Feb 15, 2023
Feb 15, 2023
Deval Mehta
Out-of-Distribution Detection for Skin Lesion DiagnosisA Summary of MICCAI 2022 paper “Out-of-Distribution Detection for Long-tailed and Fine-grained Skin Lesion Images”
Sep 15, 2022
Sep 15, 2022
Purva Natoo
Unlocking Machine Learning’s Hidden Challenge: Detecting Out-of-Distribution DataOut-of-distribution (OOD) data detection is an important problem in machine learning that involves identifying samples that are…
Oct 11, 2023
Oct 11, 2023
Raghul Asokan
Neural Networks Intuitions: 15. ASH — Paper Explanation(OOD Detection Part 2)ASH for OOD Detection
Nov 13, 2022
Nov 13, 2022
Shiro Matsumoto
Two OODs, Out of Domain and Out of DistributionTwo concepts that are often confused
Feb 7, 2024
Feb 7, 2024
Juneta Tao
Out-of-Distribution DetectionGiven a model trained on large training set, it may not perform well on the test set which is collected, e.g. different time or scenario…
May 10, 2023
May 10, 2023
In
Analytics Vidhya
by
Neeraj Varshney
Out-of-Distribution Detection in Deep Neural NetworksMaking AI systems Robust and Reliable
Dec 25, 2020
Dec 25, 2020
In
TDS Archive
by
Manish Chablani
Gradient descent algorithms and adaptive learning rate adjustment methodsHere is a quick concise summary for reference. For more detailed explanation please read:
Jul 14, 2017
1
Jul 14, 2017
1
Simon Liu
[ 技術名詞介紹 ] 知識蒸餾 Knowledge Distillation資訊揭露：此文章有經過 AI 輔助撰寫後，再進行內文修改，如有錯誤再請告知，感謝大家！
Feb 4
Feb 4
Rice Yang
對比學習 Contrastive Learning 主流方法一覽SimCLR, MoCo, SwAV, BYOL, CLIP, DeepCluster, PIRL, Barlow Twins，一覽各種模型的底層邏輯
Nov 24, 2022
2
Nov 24, 2022
2
In
TDS Archive
by
Harys Dalvi
A Fresh Look at Nonlinearity in Deep LearningThe traditional reasoning behind why we need nonlinear activation functions is only one dimension of this story.
Aug 15, 2024
8
Aug 15, 2024
8
In
The Startup
by
Er Raqabi El Mehdi
Non-Convex Optimization in Deep LearningHumans have been enjoying convex optimization (CO) for many years compared to the few contexts where they had to deal with non-convex…
Jul 28, 2020
4
Jul 28, 2020
4
In
TDS Archive
by
Lucas de Lima Nogueira
Recreating PyTorch from scratch (with GPU support and automatic differentiation)Build your own deep learning framework based on C/C++, CUDA and Python, with GPU support and automatic differentiation!
May 14, 2024
19
May 14, 2024
19
In
Secure and Private AI Writing Challenge
by
Juan Carlos Kuri Pinto
How to Make Deep Learning many Orders of Magnitude FasterDeep Learning needs no big introduction due to its wild popularity. Deep Learning or deep neural networks are a very useful set of machine…
Aug 24, 2019
Aug 24, 2019
In
azhar labs
by
azhar
Rotary Positional Embeddings: A Detailed Look and Comprehensive UnderstandingSince the “Attention Is All You Need” paper in 2017, the Transformer architecture has been a cornerstone in the realm of Natural Language…
Jan 11, 2024
7
Jan 11, 2024
7
In
Towards Dev
by
Everton Gomede, PhD
Mastering Discreteness: Enhancing Neural Network Performance with the Gumbel-Softmax DistributionAbstract
May 4, 2024
May 4, 2024
In
Mathematical Musings
by
Freedom Preetham
Mathematical Bridge Between Softmax Functions and Gibbs DistributionsThe softmax function is an essential element in various neural networks, particularly those designed for classification tasks. It…
Apr 26, 2024
2
Apr 26, 2024
2
In
TDS Archive
by
Runzhong Wang
How to Encode Constraints to the Output of Neural NetworksWondering how? Here is a systematic review of available approaches
Apr 14, 2024
Apr 14, 2024