Arion Das – Medium

Arion Das

he/him

Pinned

Published in

Towards AI

The Infamous Attention Mechanism in the Transformer architecture

THE WHY & WHEN ?

Apr 19, 2024

The Infamous Attention Mechanism in the Transformer architecture

Apr 19, 2024

Pinned

Published in

Towards AI

Advanced Attention Mechanisms — II

Flash Attention. You can refer to it’s predecessors here: KV cache, sliding window attention, MHA, MQA, uptraining, & GQA. These methods…

Nov 13, 2024

Advanced Attention Mechanisms — II

Nov 13, 2024

Pinned

Published in

Towards AI

Advanced Attention Mechanisms-I

I would recommend you go through this blog first to develop the intuition behind the infamous attention mechanism.

Nov 3, 2024

Advanced Attention Mechanisms-I

Nov 3, 2024

Pinned

Representing Words, Phrases & their Compositionality — Skip Gram Model

Representing words in a vector space helps achieve better performances on NLP tasks as it helps learning the algorithms better. I have…

Aug 26, 2024

Representing Words, Phrases & their Compositionality — Skip Gram Model

Aug 26, 2024

Pinned

Published in

Generative AI

RAGs from scratch — Why & What?!!

Ok. It’s true; LLMs are answering most of the questions out there. Gone are those days when one had to memorize repetitive stuff. LLMs are…

Feb 10, 2024

RAGs from scratch — Why & What?!!

Feb 10, 2024

Fine-tuning DeepSeek-R1 on Bhagvad Gita text chunks using Glows-AI GPUs

In this article, I’ll discuss how to use the GPU service provided by Glows.ai. It is not advisable to refer to this fine-tuning pipeline…

Feb 18

Fine-tuning DeepSeek-R1 on Bhagvad Gita text chunks using Glows-AI GPUs

Feb 18

Published in

Generative AI

Weighing Down — Subsampling & Negative Sampling

Based on the math in the skip gram model, we can identify two major drawbacks:

Aug 31, 2024

Weighing Down — Subsampling & Negative Sampling

Aug 31, 2024

Hierarchical Softmax

Softmax is the output layer function which activates the nodes in the last step of the neural network computation. It is a very popular…

Aug 27, 2024

Hierarchical Softmax

Aug 27, 2024

Published in

Generative AI

Stemming & Lemmatization

Sentence segmentation and the removal of punctuation may help in sentence-level analysis while working with textual data. But what if we…

Aug 19, 2024

Stemming & Lemmatization

Aug 19, 2024

Sentence Segmentation

This is simply, as the name suggests — breaking up the text into sentences at the appearance of full stops, questions marks, exclamation…

Aug 13, 2024

Aug 13, 2024

Arion Das

Arion Das

he/him

ri @mfsdsai, iitg || ri @aiisc || ai eng intern @ccafe || x nlp ri @oracle || naacl author || nlp || llms || dl || https://sites.google.com/view/ariondasad/home

Following

Help

Status

About

Careers

Press

Blog

Privacy

Rules

Terms

Text to speech