PinnedPublished inTowards AIAdvanced Attention Mechanisms — IIFlash Attention. You can refer to it’s predecessors here: KV cache, sliding window attention, MHA, MQA, uptraining, & GQA. These methods…Nov 13Nov 13
PinnedPublished inTowards AIAdvanced Attention Mechanisms-II would recommend you go through this blog first to develop the intuition behind the infamous attention mechanism.Nov 3Nov 3
PinnedPublished inTowards AIThe Infamous Attention Mechanism in the Transformer architectureTHE WHY & WHEN ?Apr 191Apr 191
PinnedRepresenting Words, Phrases & their Compositionality — Skip Gram ModelRepresenting words in a vector space helps achieve better performances on NLP tasks as it helps learning the algorithms better. I have…Aug 26Aug 26
PinnedPublished inGenerative AIRAGs from scratch — Why & What?!!Ok. It’s true; LLMs are answering most of the questions out there. Gone are those days when one had to memorize repetitive stuff. LLMs are…Feb 10Feb 10
Published inGenerative AIWeighing Down — Subsampling & Negative SamplingBased on the math in the skip gram model, we can identify two major drawbacks:Aug 31Aug 31
Hierarchical SoftmaxSoftmax is the output layer function which activates the nodes in the last step of the neural network computation. It is a very popular…Aug 27Aug 27
Published inGenerative AIStemming & LemmatizationSentence segmentation and the removal of punctuation may help in sentence-level analysis while working with textual data. But what if we…Aug 19Aug 19
Sentence SegmentationThis is simply, as the name suggests — breaking up the text into sentences at the appearance of full stops, questions marks, exclamation…Aug 13Aug 13
Published inGenerative AIRAGs from scratch — GenerationThe last phase of a naive RAG application — Response Generation. Generating a response using relevant document splits.Mar 12Mar 12
