ExpertsFMGPT-6 Astra is a banger - here’s everything I’ve built
How I AI · September 3, 2026
5 Selected Lessons:
case · 3 heuristic · 2
Personal recurring-task benchmarks beat generic benchmarks
Source
How I AI — GPT-6 Astra is a banger - here’s everything I’ve built (September 3, 2026) · at 0:00 ▸ · at 18:40 ▸ · at 24:22 ▸
When this applies
when evaluating any new tool or vendor release and you want a true signal of improvement rather than marketing benchmarks
The takeaway
Build your own hard, idiosyncratic test cases from your actual workflow and re-run them against every new release; sustained failure-then-success is the most reliable signal of genuine improvement.
Why it works
A task you've personally retried many times with every prior model gives you a stable baseline of difficulty; if a new model succeeds where many predecessors failed, that is strong evidence of real capability gain rather than benchmark gaming.
One level up
Trust repeated personal stress-tests over vendor-reported scores when judging whether a new capability is real.
Where it stops holding
This stops working when your personal test case is too idiosyncratic to generalize, or when a single success could be luck rather than a real capability shift.
Detail
rule: Maintain a set of hard, personally-meaningful tasks that have defeated prior models, and re-test them against every new model release.
threshold: If a model one-shots or near-one-shots a task that multiple prior frontier models failed on repeatedly for months, treat that as a genuine capability jump.
rationale: Vendor benchmarks can be gamed or abstract; your own repeated failures-turned-success are a harder-to-fake signal.
In their words
I was like, can we use Blender and get Barbie bench going? And while I will say it is imperfect, it is certainly better than it was.
Raise your ambition ceiling when the underlying tool crosses a capability threshold
Source
How I AI — GPT-6 Astra is a banger - here’s everything I’ve built (September 3, 2026) · at 24:22 ▸ · at 27:19 ▸ · at 29:28 ▸
When this applies
when your tooling improves enough that your old 'ambitious' project becomes trivial
The takeaway
Periodically re-benchmark what counts as 'ambitious' for your tools and move your project targets up accordingly, rather than anchoring to old difficulty levels.
Why it works
Once the easy version of a task (e.g., basic website vibe-coding) becomes reliably solvable, continuing to target that same difficulty level wastes the new capability headroom; shifting targets upward captures the gain.
One level up
Capability gains are only realized if you deliberately redirect ambition toward harder problems; otherwise you under-use the improvement.
Where it stops holding
This stops working when the easier task level still has unmet business value and chasing harder builds is a distraction from shipping what's actually needed.
Detail
rule: When basic builds (e.g., website vibe coding) become trivially solved, move up to more ambitious builds (3D games, desktop apps, hardware integrations) to use the new capability headroom.
threshold: If a model one-shots your previous 'hard' benchmark, treat that as the signal to redefine what 'hard' means for you.
rationale: Staying at the old difficulty level leaves capability value on the table.
In their words
Like, website vibe coding is over. We can all do that. Let's do three d games. Let's do Barbie bench.
Automated 1h45m browser QA session found real bugs with no scripted test suite
Source
How I AI — GPT-6 Astra is a banger - here’s everything I’ve built (September 3, 2026) · at 13:00 ▸
When this applies
when manual, repetitive, long-running verification work (testing edge cases, monitoring for errors) is too tedious for a human to do thoroughly
The takeaway
Delegate long, repetitive, detail-monitoring verification tasks to an agent that can sustain attention and check logs far longer than a human will tolerate.
Why it works
A computer-use agent can tirelessly repeat actions (clicking, refreshing, sending messages) while simultaneously inspecting console/error logs, catching intermittent issues like race conditions that require volume and patience to surface.
One level up
Agent labor is most valuable exactly where human attention naturally degrades — long repetitive monitoring tasks.
Where it stops holding
This stops working when bugs require domain judgment the agent lacks, or when the agent cannot interpret ambiguous failures without a human validating root cause.
Detail
situation: A newly shipped PR for chat streaming/persistence needed testing for intermittent race conditions, a task tedious and hard for a human to execute thoroughly.
decision: Asked the agent to just 'test this in Chrome'; it pulled up the preview branch, clicked through, sent chats, inspected console logs, and refreshed repeatedly.
outcome: Ran autonomously for about 1 hour 45 minutes and found and helped fix real issues, including specific race conditions.
lesson: Long-horizon, repetitive QA work is a strong fit for autonomous agent delegation.
In their words
You can see here it actually ran for an hour and forty five minutes, basically. QA ing this branch, it found a couple issues when you were navigating. It fixed some things.
Reverse-engineering an API-less hardware device (Diveoom speaker) as a running capability benchmark
Source
How I AI — GPT-6 Astra is a banger - here’s everything I’ve built (September 3, 2026) · at 18:40 ▸ · at 29:28 ▸
When this applies
when you face a closed system with no official integration path and want to test the limits of an AI coding agent's problem-solving
The takeaway
Use a genuinely hard, closed, real-world integration problem as a long-running stress test; track how successive model generations incrementally conquer it.
Why it works
Coding agents can reverse-engineer undocumented protocols (e.g., pixel-rendering algorithms) through iterative experimentation, and improved model generations progressively unlock previously-impossible hacks.
One level up
The hardest, most 'impossible' personal problems make the best longitudinal signal of real capability progress over multiple model generations.
Where it stops holding
This stops working when the target system has active anti-tampering/security measures that make reverse engineering infeasible regardless of model capability, or when legal/safety constraints prohibit hacking the device.
Detail
situation: A Bluetooth speaker with proprietary, closed software and no API resisted basic control attempts across GPT-5 and other models.
decision: Kept retrying the same hard reverse-engineering task (decode pixel algorithm, build custom app/CLI) with each new model generation.
outcome: Astra built a live-drawing app and a CLI that could stream custom messages/animations to the device, succeeding where prior models could barely get it to respond.
lesson: A fixed, hard, real-world hacking challenge is a reliable marker of genuine capability gains across model generations.
In their words
This is my my Everest. And already, I mean, the quality of what I've been able to flash onto this little device is amazing.
Previously-failed Figma automation succeeded one-shot under time pressure
Source
How I AI — GPT-6 Astra is a banger - here’s everything I’ve built (September 3, 2026) · at 27:19 ▸
When this applies
when a known tool limitation might have just been resolved by a new release, and you're under deadline pressure to find out
The takeaway
When facing a deadline and a known capability gap, retest the gap against the newest available tool before assuming you need the unavailable human expert.
Why it works
Tool limitations are time-bound to a model generation; retesting a previously-failed capability after a major release can reveal the limitation has been resolved, turning a blocked task into a solved one.
One level up
Treat documented capability gaps as provisional, not permanent, and periodically re-probe them against new releases.
Where it stops holding
This stops working when the capability gap is structural (not just a temporary model limitation) and retesting wastes time that should go to a fallback plan.
Detail
situation: Designer was unavailable before a podcast deadline and thumbnails were needed; computer-use agents had historically been poor at using Figma.
decision: Asked Astra to take over the computer and use Figma directly to build the thumbnails.
outcome: It produced usable thumbnails, succeeding at a task computer-use had previously failed at.
lesson: A new model release can silently resolve a previously well-documented capability gap.
In their words
Again, this has been something that computer use has been quite terrible at. To date, I have not gotten it to use Figma very well, but look at that. It did it.