Arion Das – Medium

Arion Das

he/him

Pinned

Published in
Towards AI

Advanced Attention Mechanisms — II

Flash Attention. You can refer to it’s predecessors here: KV cache, sliding window attention, MHA, MQA, uptraining, & GQA. These methods…

Nov 13, 2024

Advanced Attention Mechanisms — II

Nov 13, 2024

Pinned

Published in
Towards AI

Advanced Attention Mechanisms-I

I would recommend you go through this blog first to develop the intuition behind the infamous attention mechanism.

Nov 3, 2024

Advanced Attention Mechanisms-I

Nov 3, 2024

Pinned

Published in
Towards AI

The Infamous Attention Mechanism in the Transformer architecture

THE WHY & WHEN ?

Apr 19, 2024

The Infamous Attention Mechanism in the Transformer architecture

Apr 19, 2024

Pinned

Representing Words, Phrases & their Compositionality — Skip Gram Model

Representing words in a vector space helps achieve better performances on NLP tasks as it helps learning the algorithms better. I have…

Aug 26, 2024

Representing Words, Phrases & their Compositionality — Skip Gram Model

Aug 26, 2024

Pinned

Published in
Generative AI

RAGs from scratch — Why & What?!!

Ok. It’s true; LLMs are answering most of the questions out there. Gone are those days when one had to memorize repetitive stuff. LLMs are…

Feb 10, 2024

RAGs from scratch — Why & What?!!

Feb 10, 2024

Published in
Generative AI

Weighing Down — Subsampling & Negative Sampling

Based on the math in the skip gram model, we can identify two major drawbacks:

Aug 31, 2024

Weighing Down — Subsampling & Negative Sampling

Aug 31, 2024

Hierarchical Softmax

Softmax is the output layer function which activates the nodes in the last step of the neural network computation. It is a very popular…

Aug 27, 2024

Hierarchical Softmax

Aug 27, 2024

Published in
Generative AI

Stemming & Lemmatization

Sentence segmentation and the removal of punctuation may help in sentence-level analysis while working with textual data. But what if we…

Aug 19, 2024

Stemming & Lemmatization

Aug 19, 2024

Sentence Segmentation

This is simply, as the name suggests — breaking up the text into sentences at the appearance of full stops, questions marks, exclamation…

Aug 13, 2024

Aug 13, 2024

Published in
Generative AI

RAGs from scratch — Generation

The last phase of a naive RAG application — Response Generation. Generating a response using relevant document splits.

Mar 12, 2024

RAGs from scratch — Generation

Mar 12, 2024

Arion Das

Arion Das

he/him

Research Intern @Oracle | Gen-AI Research | LLMs | NLP | Deep Learning | RAGs | LinkedIn: https://www.linkedin.com/in/arion-das/

Following

Help
Status
About
Careers
Press
Blog
Privacy
Terms
Text to speech
Teams