Blog

[IMPORTANT]: Actively working on the blog. From the paper: https://arxiv.org/pdf/2501.19393 Figure 3. Budget forcing with s1-32B. The model tries to stop after “…is 2.”, but we suppress the end-of-thinking token delimiter instead appending “Wait” leading s1-32B to self-correct its answer. Interesting nuggets from s1 paper (methodology) They collected a dataset of 1k examples with reasoning traces from Google Gemini model and performed SFT (supervised fine tuning). They fix response lengths by adding “wait” tokens in certain cases to get models to generate longer CoT’s, verify, and correct itself, or halt token generation by introducing an EOT (end of thinking) token delimiter and the authors call this phenomenon as “budget forcing.” Budget Forcing Not to be picky or pedantic but budget forcing (BF) is still not a parallel inference scaling technique (as seen in o-1 or by Gemini Thinking). As the authors point out, we can think of BF as a sequential inference scaling technique. Despite the <wait> and <think tokens at appropriate steps, the model is still generating one token at a time, the only difference being in total number of tokens. ...

[IMPORTANT]: Actively working on the blog. NOTE: Although the idea behind doing something similar to “Question as a vector” ruminated in my mind for quite some time, [1] definitely played a role in fueling the fire to write this idea and present some empirical evidence. So kudos to [1] and is a great read. Background Growing up in an era where deep learning was becoming a tour de force, vectors, embeddings and representational learning was an intriguing phenomenon yet it was treated as, dare I say it, ‘fad’. Introduction to Statistical Learning was the foundation that gave me an inside overview into learning algorithms, the learning process and the broader sense of interpreting the outcomes and evaluations. Deep down for everyone, universal function approximation and neural networks seemed like a fascinating concept just because of its fantastic ability to learn over any dataset, but it seemed difficult to make the transition. ...

Graphs and LLM’s The evolving nature of experiments in artificial intelligence and the exponential pace of scientific progress in developing domain specific large language models (LLM) elevated the use cases of language models to assist researchers in advancing scientific discovery. Conversational interfaces using Auto-regressive LLM’s such as GPT-4, LLaMA, Gemini, Claude, Mistral dominated the public discourse with immediate adoption by diverse communities. Knowledge distillation from representations of LLM’s are good priors for evaluating predictive models large for AI for Science(AI4Science) initiative and scientific discovery in the age of artificial intelligence will rely on such initiatives. Scientific discovery includes several stages and collecting the data, building the experiments, analyzing the results to come up with salient hypothesis are few of the stages that have reasonable scope to include LLM’s in the loop. Augmenting various stages of the scientific process with AI models comes with a plethora of benefits and poses risks therefore it is important to make reliability, and safety of these models a priority thereby enhancing the societal benefit of scientific discovery. ...

Neural Networks Everywhere The genesis of Deep Learning began primarily in 2012 when researchers were able to capture the information present in millions of images and accurately predict class labels. This breakthrough was a culmination of multiple scientific developments. Quintessentially, Neural networks are universal functional approximators and this innate quality allows them to applied on diverse problem spaces. This is the fundamental reason behind the pervasive nature of Neural Networks. ...

I am not a philosopher but I observe the fundamental nature of things in my surroundings. Right from my childhood to this moment when I am orchestrating this article there are countless thoughts that surround my head when my inner self whispers “morality”. My thought process takes quick writhing movements wandering around searching for answers. The reason for me writing this piece is to both look for answers and portray my ethos. ...

You might be wondering what the three C’s are : Currying Closures Callbacks Ok what are those ? Before I start, let me tell you that in Javascript Functions are Objects. So every object in Javascript, be it Number, String, Array; Every Object in javascript has a prototype object. For instance if the object is declared using an object literal then, it has access to Object.prototype, Similarly all the arrays so declared have access to the Array.prototype. So since functions are objects, they can be used like any other value. They can be stored in variables, objects, and arrays. They can also be passed as arguments to functions, and do not forget that functions can be returned from functions. ...

numgo - ND Array based math library for Go Mathematical computations and statistical operations contribute immensely to the development of machine learning models in diverse disciplines. Arrays are the most frequently used data structure in most of these applications. Basing on this I have thought of porting python’s numpy library to go-lang. The name of the project is numgo. After learning go-lang I always wanted to practically implement something with the knowledge I have gathered. The project therefore, meant a lot to me since It is both a programming excercise and an opportunity to build something in go-lang. As mentioned before, the repository is inspired by numpy for python therefore, most of the concepts and ideas are broadly taken from numpy. ...

While technological and scientific advancement paved its way in the 20th century giving scope for researchers to publish of a plethora of scientific literature, “Public Understanding of Science” has been at the center stage of discussion among researchers for the past decade. This very concern sprouts from the fact that most of the scientific reportage that occurs on News Media is subject to personal interpretation. Also, the misapprehensions of the scientific findings can cause repercussions of unimaginable scale in the real world. Let us consider a small use case. For suppose, “ABC” is a research organization that publishes a scientific finding regarding the relationship between pregnancy and theobromine. Now, there might be a news media outlet that picks the same research article and broadcasts the results in a science journalism segment. The problem arises if the findings in the research paper are incorrectly represented by the Media Outlets. If the research finding says “There is a 30% chance of not having a miscarriage during pregnancy if the woman has theobromine in her diet”, the media outlet would report it in the following way, “A recent finding suggests that chocolate is good for your health during pregnancy and it would help you to reduce the chance of a miscarriage”. ...

Webcompat, is one of the recent interests that I find myself occupied with. The community is wonderful and the whole idea of having a place where users can report problems is quite fantastic. I believe that this can evolve as something greater, since webcompat.com is not only a website that focusses on bug reporting and problems, since I believe that it is a common ecosystem where users, developers and site owners are notified with the issues on the web. ...

I always used to find myself in a difficult situation when I had to travel and work. The main reason for this is my machine had a GNU/Linux based distro running and I couldn’t stay away from a charging spot for more than two hours. I always was intrigued to explore this segment and solve this very problem in order to better my travel experience. Basically there are couple of things to be kept in mind before you go out and try the tricks mentioned here, Consider them basics; So they are : ...