Quick overview · TL;DR · definition · FAQ
TL;DR
- Most "information-gathering systems" merely recreate a feed: they pull content from Hacker News, GitHub, forums, and Reddit, rank it by heat, and push it at you. It looks industrious, but really it just raises one more automated time-sink inside your phone.
- Change the metaphor and you change the fate. Don't build a feed — build an inbox. The scraper is a silent mail carrier that only delivers; the AI is an evidence-bearing reader who speaks only of what changed; the human opens the inbox only when needed — shifting from competing for attention to supplying judgment.
- The real watershed of this system isn't "how many sources it can scrape," but whether it is honest when it fails: a failure must never return a tidy "0 news items." "No news" and "didn't get the news" are two different things.
- The transferable skeleton has only four layers: sources are responsible for happening, scraping for preserving, AI for explaining, the human for judging. A good AI system shouldn't compress the world into one pretty answer; it should let the human retain the ability to return to the facts, ask again, and change their judgment.
Definition
A personal intelligence inbox is a personal information system that splits automated scraping, daily archival, AI interpretation, and human judgment into four layers. It does not pursue real-time push or sheer information volume; it pursues replayability: every conclusion can be traced back to its original source, and every scraping failure is preserved as a state rather than smoothed over in the interface into "no news today." It stands opposite the "feed" — a feed diverges (the more you look, the more there is), while an inbox converges (only when needed, find what changed).
FAQ
Is the difference between a feed and an inbox just the interaction style?
No. It decides whether a system is competing for your attention or preserving it. A feed interrupts you in real time by arrival order, forcing an instant reaction; an inbox settles by day and waits for you to read, returning to the human the right to decide when to consume information. Given the same sources, one sweeps you into the flood, the other keeps your judgment for you.
Why insist that a scraping failure must not return "0 items"?
Because "no news" and "didn't get the news" lead to opposite actions. The former means the world is quiet and you can rest easy; the latter means the system has gone blind and must re-scrape. If both are rendered as a tidy 0, you'll misread a network fault as universal peace. The value of a mature system often shows not when everything is fine, but in whether it is honest when it errs.
Why not just let the AI read everything, filter it, and draw conclusions fully automatically?
Technically you can, but "can" is not "should." The easiest mistake a personal information system makes is to outsource even the human's final power of judgment — you get a diligent assistant that judges a hundred things a day for you, only to find you haven't grown a steadier cognitive framework, just swapped in a more efficient doom-scrolling portal. Deliberately keeping three units of friction — that you must read, that you must return to the original text, that the human must ask the follow-up — is meant to make information serve the human, rather than let the human live inside the information.
References and Further Reading
- Public sources and open APIs: Hacker News, GitHub Trending/Releases, technical-forum RSS, SEC EDGAR first-hand disclosures, on-chain data interfaces.
- System-design concepts: idempotent scraping and deduplication, source health and graceful degradation, first-hand vs. second-hand evidence grading, aggregation by topic rather than a per-source list.
- Related topics on this site: long-term AI memory architecture on the command line, 24/7 automated monitoring, RAG retrieval and auditable system design.
01 · First, pinpoint the lesion: it's not that news is scarce, it's that interpretation is expensive
When most people build an "information-gathering system," their first instinct is to recreate a feed: hook up Hacker News, GitHub, X, Reddit, forums, RSS; pull the content back; rank it by heat; and push it at you. It looks very industrious, but in fact it just raises one more, more automated, time-sink inside your phone.
The system we're about to describe never set out to do that.
It is more like a mailbox. Every morning, the program quietly delivers the technical world to you; when reading is genuinely needed, the human opens the mailbox, with the AI playing the reader — laying out the changes worth noticing, the threads behind them, and what they have to do with you. The scraper takes no stance, the reader doesn't pretend to omniscience, and the user needn't respond the instant each item arrives.
A feed diverges: the more you look, the more there is. An inbox converges: only when needed, find what changed.

Something happens in the technical world every day: new models ship, open-source projects erupt, protocols are argued over, software has vulnerabilities, capital markets throw off noise, and a seemingly trivial commit may only reveal its value months later. The problem has never been a lack of content; it's that content reaches the human far faster than the human can understand it. So most people's solution becomes refreshing: refresh HN, refresh Trending, refresh the group chat, refresh the bookmarks. Information keeps piling up, while real judgment gets squeezed to last — first open it, first bookmark it, first forward it, first say "let's take a look," and then there is no "then."
This system adopts a very plain division of labor:
| Layer | What it is responsible for | What it deliberately does not do |
|---|---|---|
| Scraping layer | Fetch new items from public sources | Interpret, recommend, conclude |
| Inbox layer | Save the raw results by date | Pretend it has already understood |
| Reading layer | Filter, merge, and dig by topic | Recite every item over again |
| The human | Decide what is worth tracking further | Be forced by the system into instant reaction |
These four layers look like an engineering breakdown, but they're really a form of attention ethics. Machines are good at being punctual, repetitive, and emotionless across a hundred links; humans are good at placing a change into their own projects, interests, and long-term judgments. Blend the two, and the usual result is a machine pretending to have insight while the human is forced to refresh endlessly like a machine.
02 · A replayable pipeline: let every layer be interrogated independently
There is no complex database in this system's directory, and no swarm of agents that must run resident. At its core are just a few files: a scraping script, a dedup state, an inbox saved by date, and a cursor for "where I last read up to."
Public sources
HN · GitHub · official blogs · BTC/ETH forums · Reddit · SEC · industry RSS
│
▼
Scraping script —— fetch, light cleanup, tag source health
│
▼
inbox/2026-08-14.md —— the day's raw inbox
│
▼
Reader (AI + the human's questions) —— merge by topic, verify originals, explain impact
│
▼
Judgment / tracking list / next question
The most important design here is not "how many sources it can scrape," but that every layer can be checked independently.
If scraping fails, you can look at the log; if the daily report looks off, you can open that day's Markdown; if the AI's judgment can't be trusted, you can return to the original link; and if you go a few days without reading, nothing is silently swallowed, because the cursor records which day you last read, and the next run automatically enters a multi-day catch-up mode.
This is what replayability means. It is the exact opposite of the many AI products that "give you a result but no idea how it came about."

03 · The mail carrier's duty: dedup, keep the identity, keep the original text
Every day the scraping layer faces a motley set of sources: HN has points and comment counts, good for gauging discussion heat; GitHub Trending has stars added that day, good for gauging how fast an open-source project is erupting; official blogs and protocol forums are better for judging "whether there's any new official move"; Reddit is a community thermometer, yet easily rate-limited and drowned in memes; and a subject like SpaceX mixes industry news, market commentary, and SEC filings all at once, with wildly different fact density.
So this system does not do "let the big model summarize everything." It first does three very small but very necessary things.
First, deduplication. A post at 200 points today and 230 tomorrow doesn't deserve to reoccupy the page every day; only when the heat shows a clear jump is it marked "still gaining" or "reignited." The system doesn't try to remember all the content — it tries to remember what changed compared with last time.

Second, preserve the source's identity. An SEC 8-K, a company blog, a media report, and an analyst rating cannot sit on the same chain of evidence. They may all exist, but the reader must know: is this the company's first-hand disclosure, a media outlet's second-hand retelling, or a market participant's opinion?
Third, preserve the original link. The AI's explanation in the daily report is only an entry point, not the terminus. Once a reader is interested in an item, they should be able to return to the original discussion, the project repository, the forum thread, or the regulatory filing — not be left only to trust an unsourced summary.
These three things sound like common sense, yet many an "AI news assistant" skips exactly them: they stir first-hand material, second-hand reports, and emotional commentary into one smooth-looking pot of text. It reads great and cannot be held to account afterward.
04 · The sin of false calm: failure must not masquerade as quiet
A mature system's value often shows not when all is normal, but in whether it stays honest when it errs.
This system once hit a very typical situation: a morning DNS fault meant that HN, GitHub, Crypto, and several other sources returned nothing. In the old logic, a scraping failure and a genuine "no new content today" both returned an empty list, so the daily report showed a tidy row of zeros: HN 0, GitHub 0, Crypto 0.
This is a dangerous false calm.
"No news" and "didn't get the news" are not the same thing.
The reworked system adds a three-state health mark to every source:
| State | Meaning | What the daily report should say |
|---|---|---|
ok | Scraping and parsing succeeded; there may simply be no new items | "No new content" |
partial | Some endpoints succeeded, some failed | "Partial success," and list the missing sources |
failed | The source produced no valid content this round | "Scraping failed" — never written as 0 news items |
When several core sources fail at once, the daily report is flagged outright as "scraping incomplete," and the script exits with a non-zero status; seeing this flag, the reader does not advance the reading cursor and does not interpret the blank as the world having gone quiet. Only after the network recovers and a re-scrape runs does the inbox count as truly delivered.
This is a very plain principle in system design: uncertainty must be preserved as data, not smoothed over in the interface.

05 · Grading the evidence: open a separate channel for first-hand disclosures
Take one company's capital-markets intelligence as an example. The news might carry, all at once, "analysts raise the rating," "the share price returns to a certain level," "dense launches," and "an 8-K filing." These headlines crowd together and look like a single hot topic, but they are not the same kind of thing.
Launch count is an operational fact; an analyst rating is an opinion; a share-price move is a market outcome; and 10-Q, 10-K, 8-K, and 13D/13G are regulatory disclosures. Their weight, timeliness, and verifiability all differ.
So the system handles this kind of source with great restraint:
- 10-Q and 10-K count as high-signal, earnings-class filings;
- an 8-K is roughly sorted by its body into results, financing/debt, or other material events;
- a 13D/13G counts as a change in major shareholders — it does not render an ordinary holdings filing as a big operational boon;
- Form 3/4/5 is de-noised and archived by default;
- a filing must not merely appear in the index — its body must be retrievable and matched to the correct issuing entity before it enters the daily report.
That last step matters especially. A data source returning "there is a 10-Q" does not mean the system has actually read that 10-Q. A dead link, an index delay, a permission denial, or a mismatched corporate entity can all turn a seemingly authoritative message into a false signal. Better for the daily report to say "the body cannot yet be verified" than to fabricate a conclusion from a filing that doesn't exist.
This, too, is the common sense the AI era most needs re-emphasized: credibility comes not from a confident tone, but from a chain of evidence that can be re-checked.
06 · The art of reading: speak of change by topic, not read out a menu by source
The raw inbox is still saved by source, because the machine needs to know clearly where its inputs came from; but the reading finally handed to the human should no longer be lined up by HN, Reddit, GitHub, like reciting a shopping list.
What the human truly cares about is the topic:
- what happened with AI models and agent tooling;
- whether some open-source ecosystem has taken a new direction;
- whether protocol-layer debates have any bearing on long-term judgment;
- whether a company's operations, financing, and market narrative are diverging;
- which seemingly lively items are, in fact, just repeated retellings.
So the final output is organized by topic, and for the key topics it generates very short "signal cards": what the new fact is, which parts are second-hand noise, and what to verify next. It does not make investment decisions for you, nor decide your stance; it only makes the next question more precise.
For instance, a "capital-markets radar" needn't repeat every share-price comment daily. It need only say: today there were this many first-hand disclosures, this many operational facts, this many market opinions; the next verification point is the next 10-Q, a certain material 8-K, or an actual business milestone. That way the human isn't led by the hot topic, but can place the hot topic into their own verification framework.
07 · Keep three units of friction: why it doesn't chase full automation
Reading this far, someone may ask: why not let the AI read everything, filter automatically, push automatically, and conclude automatically? Technically you can, of course.
But "can" is not "should."
The easiest mistake a personal information system makes is to outsource even the human's final power of judgment. You get a seemingly diligent assistant: it tells you ten things a day, twenty, a hundred, and even judges for you which ones matter. Yet after a while you find you haven't formed a steadier cognitive framework — you've just swapped in a more efficient doom-scrolling portal.
So this system deliberately keeps three units of friction:
- it waits for you to read, rather than interrupting you endlessly;
- it returns to the original text, rather than consuming only the summary;
- the follow-up question is issued by the human, rather than the model finishing the world's story on the human's behalf.
Its goal is not to let you live inside information, but to let information serve you when needed.
08 · Drawing the general from the particular: where this skeleton can grow
It suits far more than technical news. It is really a transferable "private intelligence desk" pattern.
| Scenario | What the scraping layer can hook up | What the reading layer should ask |
|---|---|---|
| Open-source teams | Issues, releases, dependency vulnerabilities, PRs | Which changes affect the roadmap? |
| Investment research | Company disclosures, industry data, earnings calls, regulatory filings | Does the new fact change the original thesis? |
| Content teams | RSS, competitor updates, reader feedback, search trends | Which topic is worth writing, rather than which word is hottest? |
| Household and personal | Calendar, health data, bills, memos | Which changes require action right now? |
The underlying structure never changes: sources are responsible for happening, scraping for preserving, AI for explaining, the human for judging.
A genuinely good AI system shouldn't compress the world into one pretty answer; it should let the human retain the ability to return to the facts, ask again, and change their judgment.
09 · Coda: a companion that keeps the mailbox for you
Strip this system to the bone and it is really quite small — just a pile of Markdown, a few RSS feeds, a batch of public APIs, plus a human willing to ask questions. But what it points toward is perhaps a more worthwhile kind of relationship with AI: not an assistant forever talking, but a companion that keeps the mailbox for you, holding the world's changes until you are ready to open them.
Perhaps the most valuable thing about AI is not helping us see more.
It is helping us see a little less, yet see it more accurately.