*"This is a 1 hour general-audience introduction to Large Language Models: the core technical component behind systems like ChatGPT, Claude, and Bard. What they are, where they are headed, comparisons and analogies to present-day operating systems, and some of the security-related challenges of this new computing paradigm."*

Andrej Karpathy - Intro to Large Language Models [Video]

Hi I'm Jeremy. I'm building this site because I'm not satisfied with other online community type websites and their lack of innovation (see the /about page). The goal is to make a superior, more open, and more decentralized online community.

The site is still in early stages in active development (see the dev masterthread here: https://zsync.xyz/p/zsync-dev-update-masterthread)

Join us if this sounds interesting! Feel free to send me any feedback or reach out directly!

*"While self-correction has shown promise in improving LLM outputs in terms of style and quality (e.g. Chen et al., 2023; Madaan et al., 2023), recent attempts to self-correct logical or reasoning errors often cause correct answers to become incorrect, resulting in worse performances overall (Huang et al., 2023). In this paper, we break down the self-correction process into two core components: mistake finding and output correction. For mistake finding, we release BIG-Bench Mistake, a dataset of logical mistakes in Chain-of-Thought reasoning traces. We provide benchmark numbers for several state-of-the-art LLMs, and demonstrate that LLMs generally struggle with finding logical mistakes. For output correction, we propose a backtracking method which provides large improvements when given information on mistake location. We construe backtracking as a lightweight alternative to reinforcement learning methods, and show that it remains effective with a reward model at 60-70% accuracy."*

LLMs cannot find reasoning errors, but can correct them

What I believe to be one of the most underrated features of LLMs is the ability to convert imprecise human text into code.

In the past, writing a computer program required writing code - aka writing text in a very specific, tedious way such that it can be understood by a computer. A single erroneous character will crash an entire program. Writing code is difficult and tedious enough that it is a professional career that tends to require some degree of study (eg. 4 year CS degree, coding bootcamp), and the job can command annual compensation packages from $100k-$1m+/yr.

LLMs for the first time enable writing programs through natural language, ie. without writing code.

What this will do is make it dramatically easier to create computer programs. The technological jump is the equivalent of trying to talk to someone in Icelandic in 1989 (assuming you don't speak Icelandic), and then suddenly being given access to a modern smartphone with Google Translate.

Some implications:

- Anyone will be able to easily create or update computer programs
- Computer programs will easily be able to create computer programs
- Coding for the first time in history will just be done by regular human language instructions that any non-technical person can write. A disgruntled customer or product manager will simply be able to create a bug report or feature request, and an AI agent will immediately be able to take that and update the codebase (eg. submitting a pull request after testing and code reviewing its own work)
- Programming languages will become unimportant, because any language will easily be able to be translated into any other
- Coding will be commoditized

The human programmer is the translation layer between the human and the machine, and AI is automating that work away, just like it's automating human language translators.

*AI skeptic: So what? Who cares if some geeks can make computer programs easier?*

AI and automation are computer programs!

**AI is making it easier to improve AI.**

This is a feedback loop that results in exponential growth of AI. The more AI improves, the more AI is able to improve itself. Eventually this feedback loop gets so out of control that humans cannot keep up, a point known as the "singularity".

Humans are terrible at comprehending exponential growth, and thus I believe are drastically underestimating how quickly this is all developing.

Most seem to think that the job market for most people is going to be fine over the next decade - I vehemently disagree. I believe that AI will dramatically change society over the next few years and make the workforce drastically more competitive as AI starts to outstrip human ability. The labor market will become so competitive that UBI will become inevitable (I've been saying this for a decade, but never before has it looked so obvious and imminent).

The nature of work will move to a higher level of abstraction. Workers will become more like managers who delegate the "grunt work" to AI - responsible less for the nitty gritty of implementation and building tedious things, and responsible more for deciding what to build and where to direct the AI agents' focus. Just like fast mental math skills hasn't really been a competitive advantage since the invention of the calculator, doing any sort of repetitive, automatable work will not be of any advantage because the AI will be able to do it faster and cheaper at a speed of gigabytes/s, while being able to self-correct and debug itself. As tech becomes commoditized, humans will have to focus more on the "human" aspect such as marketing and zero sum games to economically compete.

The mistake that people who dismiss this tend to make is that they often believe that AGI is required for this mass societal transformation. I disagree. All it takes is for computer programs to be able to make the creation of computer programs easier to achieve this AI flywheel effect.

Anyways just some random thoughts here at 2:11am in Seoul.

LLMs create an Exponential Flywheel for AI Advancement

HN: <https://news.ycombinator.com/item?id=37148210>

LlamaGPT – Self-hosted, offline, private AI chatbot, powered by Llama 2

Show HN: <https://news.ycombinator.com/item?id=36958731>

Magic Loops – Combine LLMs and code to create simple automations

"AutoChain takes inspiration from LangChain and AutoGPT and aims to solve both problems by providing a lightweight and extensible framework for developers to build their own agents using LLMs with custom tools and [automatically evaluating](https://github.com/Forethought-Technologies/AutoChain#workflow-evaluation) different user scenarios with simulated conversations. Experienced user of LangChain would find AutoChain is easy to navigate since they share similar but simpler concepts."

AutoChain, lightweight and testable alternative to LangChain

<https://news.ycombinator.com/item?id=36641393>

llm

t/llm