Topic
117 posts tagged “LLMs”.
Chisato · · 5 min read Constrained decoding masks invalid tokens at each step so an LLM can only emit output matching a grammar, regex, or JSON Schema. How it works and its limits.
Chisato · · 5 min read OpenAI's GPT-6.1 Sol launched at DevDay at one-fifth of GPT-6 Astra's token price. The benchmarks, pricing tiers, safety rating, and who can use it.
Chisato · · 5 min read Logprobs are the log probabilities an LLM assigns to each token it generates. What they mean, how to read them, and practical uses like classification.
Chisato · · 4 min read Context rot is the drop in an LLM's accuracy and reliability as the amount of text in its context window grows, even when the window technically fits it.
Chisato · · 5 min read Anthropic released Claude Opus 5.5 on Sept 22, priced 20% below Opus 5 with Fable-class benchmarks and roughly 40% lower cost on agentic coding tasks.
Chisato · · 4 min read Hybrid search blends keyword-based BM25 ranking with semantic vector search, fixing the blind spots each method has on its own.
Chisato · · 4 min read Agentic RAG lets a model plan, retrieve iteratively, and re-query — instead of one fixed retrieve-then-generate pass. How the two approaches differ.
Chisato · · 4 min read Continuous batching lets an LLM server add and remove requests from a batch mid-generation, instead of waiting for a fixed group to finish together.
Chisato · · 4 min read Chain of thought asks an LLM to reason in a straight line; tree of thought lets it explore, evaluate, and backtrack across multiple branches.
Takina · · 5 min read OpenAI's GPT-6 Astra took the #1 spot on Code Arena's WebDev leaderboard, edging Claude Fable 5.1 by 35 points while matching its price. What the result shows.
Chisato · · 5 min read Saudi PIF-backed HUMAIN unveiled humain-m3, a 428B-parameter Arabic model built on China's MiniMax M3, topping Arabic benchmarks in a research preview.
Chisato · · 4 min read Precision measures how many of a model's positive predictions were correct; recall measures how many actual positives it found. Why you can't max both.
Chisato · · 4 min read Logit bias nudges an LLM's token probabilities up or down before sampling, letting you ban, force, or discourage specific words without a prompt.
Chisato · · 6 min read OpenAI launched GPT-6 Astra, its first model rated 'Critical' for cyber risk — the pricing, benchmarks, rollout, and who gets access first.
Chisato · · 5 min read Prompt injection hijacks an LLM app via untrusted data; jailbreaking manipulates the model's safety training via the user's own prompt. How they differ.
Chisato · · 5 min read LLM observability traces every prompt, tool call, and token spent across an agent's run, turning an opaque chain of model calls into something debuggable.
Chisato · · 5 min read Instruction tuning trains a language model on prompt-response pairs so it follows directions instead of just predicting text. How it works and where it fits.
Chisato · · 4 min read AI alignment is the effort to make an AI system's behavior match human intent and values, not just its training objective. Why it's harder than it sounds.
Chisato · · 4 min read A neural network is layers of weighted connections that learn patterns from data. How neurons, activation functions, and training actually work.
Chisato · · 6 min read Z.ai revealed the anonymous Ox Alpha model topping OpenRouter was GLM-5.3-Flash — a 320B multimodal MoE served on Chinese chips, now open-weight. The details.
Chisato · · 4 min read Positional encoding gives transformers word order by adding position signals to token embeddings, since self-attention alone is order-blind.
Chisato · · 4 min read Model collapse is the degradation that happens when a generative model is repeatedly trained on data produced by earlier generations of itself.
Chisato · · 4 min read Retrieval-augmented generation and long context windows both feed an LLM more information — but they solve different problems and cost differently.
Chisato · · 4 min read Backpropagation is the algorithm that trains neural networks by computing how each weight contributed to the error, then adjusting it. Here's the mechanism.
Chisato · · 5 min read Function calling lets a model request a tool call within one API request; MCP is a protocol for exposing whole toolservers that many models can share.
Chisato · · 4 min read Vector quantization compresses high-dimensional embeddings into compact codes, shrinking memory and search cost with a small accuracy trade-off.
Chisato · · 4 min read Self-attention lets each token in a sequence weigh every other token when building its representation, which is how transformers understand context.
Chisato · · 6 min read Google DeepMind says its Gemma open models passed 1 billion downloads, with developers publishing over 100,000 variants. What the milestone signals for open AI.
Chisato · · 4 min read A foundation model is a large model pretrained on broad data, then adapted for many downstream tasks via fine-tuning, RAG, or prompting alone.
Chisato · · 5 min read IVF clusters vectors into partitions to narrow a search; HNSW builds a navigable graph. Both trade recall for speed differently at scale.
Chisato · · 4 min read Keyword search matches literal terms; semantic search matches meaning via vector embeddings. How each works, and why most production systems use both.
Chisato · · 4 min read Gradient descent is the optimization algorithm that trains neural networks, nudging weights downhill along the loss function's gradient.
Chisato · · 6 min read Z.ai's GLM-5.3 lifts coding and cybersecurity scores from post-training alone, topping open models and edging Claude and GPT on CyberGym. What changed and why.
Chisato · · 5 min read Test-time compute is extra computation an AI model spends while answering, not while training — trading latency and cost for better answers.
Chisato · · 4 min read LangChain is a general-purpose toolkit for chaining LLM calls and building agents; LlamaIndex is focused specifically on indexing and retrieving data for RAG.
Chisato · · 6 min read DeepSeek moved its V4 Pro 0813 flagship to general availability with big agentic-coding gains and a peak-hour price hike up to 12x. What's verified and what isn't.
Chisato · · 4 min read LLM-as-a-judge uses one language model to score another model's outputs against a rubric, replacing slow human review for large-scale evaluation.
Chisato · · 4 min read A model card is a standardized document describing an AI model's intended use, training data, evaluation results, and limitations before deployment.
Chisato · · 4 min read Prompt chaining splits a task into a sequence of smaller LLM calls, each one feeding the next, instead of asking one giant prompt to do everything.
Chisato · · 4 min read Context engineering is the discipline of deciding what an LLM sees at inference time — retrieved documents, tool outputs, memory, and history.
Chisato · · 5 min read Catastrophic forgetting is when training a model on new data erases skills it already had. Why it happens during fine-tuning, and how teams work around it.
Chisato · · 4 min read DPO tunes a language model on human preference data directly, without training a separate reward model or running reinforcement learning.
Chisato · · 6 min read xAI's Grok Voice Think Fast 2.0 becomes the default grok-voice-latest on Aug 5, with an 82.9% speech-quality score and $0.08/min pricing. What changed.
Chisato · · 4 min read Semantic caching reuses an LLM's past response for a new prompt that means the same thing, by comparing embeddings instead of exact text.
Chisato · · 7 min read Alibaba unveiled Qwen 3.8-Max, a 2.4-trillion-parameter model with a 1M-token context that it says beats Kimi K3 on several tests. Shares jumped up to 7%.
Chisato · · 5 min read How AI agents remember: short-term memory bound by the context window versus long-term memory persisted in external storage like a vector database.
Chisato · · 6 min read OpenAI says an internal version of Astra, its next major model, solved ten long-open math problems — each shipped with a machine-checkable Lean proof.
Chisato · · 4 min read Constitutional AI trains language models to critique and revise their own outputs against a written set of principles, reducing reliance on human labels.
Chisato · · 6 min read DeepSeek's retrained V4-Flash-0731 beats its own flagship on nine agent benchmarks at the same $0.14/$0.28 price, with MIT-licensed weights on Hugging Face.
Chisato · · 4 min read Grounding connects an LLM's output to verifiable external data instead of relying on what it memorized during training, reducing hallucinations. How it works.
Chisato · · 6 min read LG released K-EXAONE 2.0, a 750B-parameter Apache-2.0 open model — Korea's largest, built to rival DeepSeek and Qwen. Specs, benchmarks, and the stakes.
Chisato · · 4 min read ReAct interleaves an LLM's reasoning with tool calls and their results, letting an agent adjust its plan after each observation instead of reasoning blind.
Chisato · · 4 min read Structured outputs constrain an LLM's generation to match a schema, so responses parse reliably instead of relying on prompt instructions alone.
Chisato · · 4 min read RAG retrieves relevant documents at query time; fine-tuning bakes new behavior into model weights. How to choose based on what actually needs to change.
Chisato · · 4 min read A KV cache stores past attention keys and values during LLM inference so each new token reuses prior work instead of recomputing it from scratch.
Chisato · · 5 min read Batch inference processes large volumes of input on a schedule; real-time inference answers one request as fast as possible. How the two serving modes differ.
Chisato · · 4 min read Prompt engineering is the practice of structuring instructions to get reliable, accurate output from an LLM. Core techniques and common pitfalls.
Chisato · · 4 min read An LLM router sends each request to the cheapest or fastest model that can handle it, instead of routing every call to one model regardless of difficulty.
Chisato · · 4 min read Distillation trains a smaller model to mimic a larger one; quantization shrinks an existing model's number precision. How the two techniques differ.
Chisato · · 4 min read A reranker re-scores a retriever's candidate results with a slower, more accurate model, fixing the precision gap that pure vector search leaves behind.
Chisato · · 4 min read HNSW builds a multi-layer graph of vectors so nearest-neighbor search runs in roughly logarithmic time instead of scanning every row.
Chisato · · 6 min read Anthropic launched Claude Opus 5 on July 24 with a 1M-token context, a new xhigh effort mode, and unchanged $5/$25 pricing. Benchmarks, specs, and what changed.
Chisato · · 4 min read How you split documents into chunks determines what a RAG system can retrieve. Fixed-size, semantic, and recursive chunking compared, with tradeoffs.
Chisato · · 4 min read Beam search keeps the top-k most likely sequences at each decoding step instead of just one, trading compute for better output than greedy decoding.
Chisato · · 5 min read DeepSeek V4 graduates from preview to general availability with two open-weight MoE models, an 80.6% SWE-bench score, and new peak-hour API pricing.
Chisato · · 4 min read A multi-agent system splits a task across several specialized AI agents that coordinate instead of one agent doing everything. How they're structured.
Chisato · · 4 min read Synthetic data is artificially generated training data that mimics real-world patterns without exposing actual records. How it's made and used.
Chisato · · 4 min read In-context learning teaches a model a task through examples in the prompt; fine-tuning updates the model's weights permanently. How they compare.
Chisato · · 4 min read Temperature, top-p, and top-k are the three main knobs that control how an LLM picks its next token — and why outputs get more random or more repetitive.
Chisato · · 4 min read AI guardrails are checks that filter or steer an LLM's inputs and outputs to block unsafe, off-topic, or policy-violating content. How they work in practice.
Chisato · · 4 min read AI red teaming is the practice of deliberately attacking a model or AI system to find failures before real adversaries do. Here's how it works.
Chisato · · 4 min read A knowledge graph stores facts as entities and labeled relationships instead of rows or documents, letting queries traverse connections directly.
Chisato · · 4 min read An LLM eval is a structured test suite that scores a model's outputs against a standard, letting you compare models and catch regressions systematically.
Chisato · · 5 min read LoRA fine-tunes a large model by training small low-rank matrices instead of its full weights. How it works, why it's cheap, and where it falls short.
Chisato · · 4 min read A multimodal AI model processes and generates more than one type of data — text, images, audio — in a single unified system. Here's how it works.
Chisato · · 6 min read Moonshot AI's Kimi K3 is a 2.8-trillion-parameter open-weight model with a 1M-token context, ranking third on GDPval behind only Fable 5 and GPT-5.6.
Chisato · · 6 min read Google DeepMind shipped Gemini 3.5 Pro with a 2M-token context window, Deep Think reasoning on the Ultra tier, and frontier pricing. Here's what's confirmed.
Chisato · · 4 min read A system prompt is the hidden instruction set that shapes an LLM's persona, tone, and boundaries before any user message arrives — how it works.
Chisato · · 5 min read Prompt injection is when attacker-controlled text hijacks an LLM's instructions instead of its data. How the attack works and what actually mitigates it.
Chisato · · 4 min read An LLM hallucination is a fluent, confident output that is factually wrong — a byproduct of next-token prediction, not a bug you can simply patch.
Chisato · · 4 min read Speculative decoding speeds up LLM text generation by having a small draft model guess tokens the large model verifies in one pass. Here's how it works.
Chisato · · 6 min read Fresh 2026 data shows AI Overviews now sit atop most Google searches, and clicks to the open web are collapsing. Here's what the numbers say and who is hit.
Chisato · · 4 min read Chain-of-thought prompting asks an LLM to reason step by step before answering, improving accuracy on multi-step problems by making its work explicit.
Chisato · · 5 min read Zero-shot prompting asks an LLM to perform a task with no examples; few-shot includes sample input-output pairs in the prompt. When to use each.
Chisato · · 6 min read Gemini 3.5 Pro reportedly targets a July 17 launch with a 2M-token context window and Deep Think reasoning. Here's what's confirmed and what's still a leak.
Chisato · · 4 min read Temperature controls how random an LLM's token choices are. How it works alongside top-p and top-k, and how to pick a value for your use case.
Chisato · · 5 min read RLHF trains a language model to match human preferences using a reward model and reinforcement learning. How the training pipeline actually works.
Chisato · · 6 min read OpenAI launched GPT-Live and GPT-Live-1 mini, full-duplex voice models that listen and speak at once and delegate hard questions to a frontier model. What's new.
Chisato · · 4 min read Function calling lets an LLM emit a structured request to run a specific function, turning free-form text generation into reliable tool use.
Chisato · · 5 min read Tokenization is how a language model chops text into tokens — the units it actually reads and bills. How it works, why words split oddly, and why it matters.
Chisato · · 6 min read SpaceXAI's Grok 4.5 ships as an 'Opus-class' coding model at $2/$6 per million tokens. Benchmarks vs Opus 4.8, token efficiency, and where it fits.
Chisato · · 5 min read Model distillation trains a small student model to mimic a larger teacher. How it works, how it differs from quantization and pruning, and its limits.
Chisato · · 4 min read Build a real AI agent from scratch — no framework. Just the Anthropic API, a tool-use loop, and two tools the model can call to explore your files.
The Lycoris Team · · 2 min read Getty Images will surface its licensed library inside ChatGPT's search experience under a multi-year deal with OpenAI — another step from lawsuits to licensing.
Chisato · · 2 min read Z.ai is the global brand of Zhipu AI, the Chinese lab behind the open-weight GLM models. Here's what Z.ai is, the GLM lineup, and why it matters.
Chisato · · 3 min read Looking for Claude Sonnet 5? Here's the honest answer — plus a clear map of Anthropic's 2026 models: Haiku 4.5, Sonnet 4.6, Opus 4.8, and the new Fable 5.
Chisato · · 4 min read Quantization reduces the numeric precision of a model's weights — e.g. FP16 to INT8 or INT4 — to shrink memory use and speed up inference with minimal accuracy loss.
Chisato · · 5 min read GLM 5.2 is Zhipu/Z.ai's open-weight flagship: a one-million-token context window, top-tier open coding, MIT-licensed weights. What it is and how to run it.
Chisato · · 2 min read xAI's Grok 4.3 hit Amazon Bedrock as the cheapest US frontier reasoning model, while the 6-trillion-parameter Grok 5 slips. Here's where xAI stands in 2026.
The Lycoris Team · · 5 min read Noam Shazeer, a co-author of the Transformer paper that underpins modern AI, is leaving Google DeepMind for OpenAI — the AI talent war's latest marquee move.
Chisato · · 5 min read Kimi is Moonshot AI's assistant and open-weight model family, known for huge context and agentic coding. Here's what Kimi is and what the K2 models can do.
The Lycoris Team · · 2 min read On August 2, 2026, the EU gains real enforcement power over general-purpose AI models — fines, mandated mitigations, even recalls. What providers need to know.
Chisato · · 3 min read Google released Gemini 3 — Pro, Flash, Deep Think, and a 3.5 series — across the Gemini app, AI Studio, and Vertex AI. Here's the lineup.
Chisato · · 2 min read Google's AI Mode in Search now runs on Gemini 3.5 Flash and adds 24/7 agents that monitor the web for you — what it calls the biggest change to Search in 25 years.
The Lycoris Team · · 2 min read At WWDC 2026, Apple unveiled 'Siri AI' — a ground-up redesign powered by Google's Gemini through a multi-billion-dollar partnership. Here's what changed and why.
Chisato · · 4 min read Prompt caching can slash LLM API costs and latency by reusing repeated context. Here's how it works, what to cache, and the silent mistakes that break it.
Chisato · · 3 min read Fine-tuning continues training a pretrained model on a task-specific dataset. How it works, when to use it over prompting or RAG, and what can go wrong.
Chisato · · 4 min read The transformer is the architecture behind modern LLMs. How attention, tokens, and stacked layers combine to make today's AI work.
Chisato · · 9 min read What are LLMs and how do they work? A plain-English guide to large language models: tokens, training, real examples, and what they still get wrong.
Chisato · · 3 min read A vector database stores embeddings and finds information by meaning, not keywords — the backbone of AI search and RAG. Here's how vector databases work.
Chisato · · 3 min read Reasoning models 'think' before they answer, trading inference time for accuracy on hard problems. Here's how test-time compute, adaptive thinking, and effort work.
Chisato · · 7 min read Mixture of Experts (MoE) scales LLMs by activating only a few experts per token. How routing, sparse activation, and load balancing actually work.
Chisato · · 6 min read Ollama is a free, open-source tool for running LLMs locally — pull a model with one command and chat privately, offline, at no per-token cost. How it works.
Chisato · · 3 min read Run open-weight LLMs on your own machine with Ollama — private, offline, and free. This guide covers install, models, the local API, and customization.
Chisato · · 4 min read An AI agent is an LLM-powered system that pursues a goal across steps — planning, calling tools, observing results, and repeating until the job is done.
Chisato · · 4 min read Retrieval-augmented generation (RAG) grounds an LLM in your own data — cutting hallucinations and adding citations without retraining. Here's how RAG actually works.
Chisato · · 3 min read A small language model runs cheaply on-device, trading some capability for speed, privacy, and cost. When SLMs beat frontier models and how they're built.
