How to Make Your AI Smarter

I hear some AI enthusiasts say things like, “The more I use ChatGPT, the smarter it gets, because it gets to know me and learns as it goes.”

This isn’t quite right. While it’s true that AI assistants learn things about you and your preferences and save them to memory, that doesn’t do much to actually change (or improve) the behavior of the chatbot. It’s not actually “getting smarter.”

That said, there is a way to make your AI smarter, and that’s to tell it what to do differently. Change its instructions.

I have my own way of doing this that I can “SCIP”: the Socratic Continuous Improvement Protocol. Here’s how it works.

Take vibecoding with ChatGPT Codex and Claude Code. Sometimes, I have to walk the AI step-by-step through how I want it to behave. Other times, it figures it out on its own. Either way, when it gets something right that it’s not doing by default, I make a point to capture the insight and add it to the instructions.

As an example, below is an exchange I had with Codex and Claude Code.

The bottom line: When your AI gets something right, capture that insight and add it to the instructions—turning one-off successes into permanent behavior. When ChatGPT Codex correctly diagnosed that the best way to fix my website problem was to edit not the website code but the exporter itself, I documented that decision-making logic so future AI agents would know to do the same.

look at Socratic.

it seems like the claude theme is getting applied to my intro. even though in my org file, i don’t invoke the claude theme until later.

what is happening?

- Explored
└ Read the-myth-of-sycophancy.mdx, 20251126140335-the_myth_of_sychophancy.org
Search theme in ox-astro-helpers.el
Read ox-astro-helpers.el, ox-astro-handlers.el
Search plist-put info :theme
Search THEME in ox-astro-helpers.el

75 more lines

- Explored
└ Read the-myth-of-sycophancy.mdx, 20251126140335-the_myth_of_sychophancy.org
Search theme in ox-astro-helpers.el
Read ox-astro-helpers.el, ox-astro-handlers.el
Search plist-put info :theme
Search THEME in ox-astro-helpers.el
Search theme: in the-myth-of-sycophancy.mdx

- Edited ~/Dropbox/github/ox-astro/ox-astro-helpers.el (+17 -3)
1526 (org-trim v))))
152\1- (theme (let ((th (plist-get info :theme)))
152\1- (when (and th (not (string-empty-p (org-trim th))))
152\1- (org-trim th))))
1527 + (theme
1528 + (let* ((th (plist-get info :theme))
1529 + (theme-clean (and th (not (string-empty-p (org-trim th))) (org-trim th)))
1530 + (first-headline-pos (org-element-map tree 'headline
1531 + (lambda (h) (org-element-property :begin h))
1532 + nil 'first-match))
1533 + (first-theme-pos (org-element-map tree 'keyword
1534 + (lambda (k)
1535 + (when (string-equal (org-element-property :key k) "THEME")
1536 + (org-element-property :begin k)))
1537 + nil 'first-match)))
1538 + ;; If the first THEME keyword appears after the first heading, treat it as inline
1539 + ;; only and avoid promoting it to page/frontmatter theme.
1540 + (if (and theme-clean first-theme-pos first-headline-pos
1541 + (>= first-theme-pos first-headline-pos))
1542 + nil
1543 + theme-clean)))
1544 (status (plist-get info :status))

- Ran cd /Users/jay/Dropbox/github/ox-astro && git status -sb
└ ** main...origin/main [ahead 2]
M ox-astro-helpers.el

- Ran cd /Users/jay/Dropbox/github/ox-astro && emacs -batch \
│ -eval "(setq load-path (cons \"/Users/jay/Dropbox/github/ox-astro\" load-path))" \
│ -eval "(require 'org)" \
│ ... +6 lines
└ (no output)

- Explored
└ Read the-myth-of-sycophancy.mdx

- Edited ~/Dropbox/github/ox-astro/docs/work-log.org (+9 -0)
49
50 +* 2025-12-04
51 +** Session: Prevent inline THEME from setting page theme
52 +
53 +*** Summary
54 +- Only promote THEME to frontmatter when it appears before the first heading; inline theme markers now stay inline and no longer set page-w
ide theme
55 +
56 +*** Files Modified
57 +- ox-astro-helpers.el
58 +
59 * 2025-11-30

- Ran cd /Users/jay/Library/CloudStorage/Dropbox/github/astro-monorepo && git status -sb
└ ** main...origin/main
M apps/socratic/src/content/blog/the-myth-of-sycophancy.mdx

─ Worked for 2m 52s ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

- Root cause: the inline #+THEME: claude was being promoted to frontmatter because ox-astro treated any THEME keyword as page theme. That made the
whole page render as Claude before the inline marker.

Fix applied (not pushed):

- ox-astro now only promotes a THEME keyword to frontmatter if it appears before the first heading; otherwise it stays inline. (/Users/jay/Dropbox/
github/ox-astro/ox-astro-helpers.el)
- Re-exported the note so the MDX frontmatter no longer sets theme:; inline theme markers remain. (apps/socratic/src/content/blog/the-myth-of-
sycophancy.mdx)
- Logged the change in ox-astro work log. (/Users/jay/Dropbox/github/ox-astro/docs/work-log.org)

*Status:*

- Astro repo has one modified file pending: apps/socratic/src/content/blog/the-myth-of-sycophancy.mdx (frontmatter no longer forces Claude).
- ox-astro repo has pending changes in ox-astro-helpers.el and docs/work-log.org.
- Nothing pushed per your instruction. I can commit/push when you say the word.

I asked you about a problem with my astro theming, and you correctly understood that to fix the problem, you needed to change the behavior of my org-to-astro exporter, ox-astro! That was perfect. It was the correct decision. Is that logic documented somewhere, or did you just figure out that was the right thing to do?

It wasn't documented - I inferred it from the symptom (page-wide Claude theme before the inline marker) and the exporter's behavior of promoting any THEME keyword into frontmatter. That pointed to ox-astro as the right place to fix it by only promoting THEME if it appears before the first heading.

well it was perfect. let’s add that logic / protocol to the instructions for Next Codex to enshrine / cement/ codify this desireable behavior. Where should we document it?

- We should enshrine it in ox-astro's own docs so future exports follow the rule: 
- Add a short "Theme promotion" note to ox-astro/docs/instructions.org (or docs/design-architecture.org if you prefer) stating: "Promote THEME to frontmatter only if it appears before the first heading; THEME after the first heading is inline only and must not set page-wide theme." 
- Optionally drop a brief entry in docs/work-log.org or docs/codebase-wisdom.org (ox-astro) summarizing the fix and rationale.

Want me to add it to ox-astro/docs/instructions.org and a quick note in docs/codebase-wisdom.org?

what? no. not theme promotion. I mean enshrine the idea that sometimes to fix a problem with Astro, we sometimes need to edit the actual org-to-mdx exporter, not the Astro itself.

and i mean enshrine it in my astro repo docs.

you missed the point. We are in astro repo now. i asked you to fix something. and you figured out that you had to edit a different codebase, in a different repo. that’s the insight we need to enshrine in the astro monorepo AGENTS.md

Got it. We should document in the astro monorepo (AGENTS.md or instructions.org) that when frontmatter/theme/markdown behavior is wrong, the right fix may be in the org→MDX exporter (ox-astro), not in Astro templates. I'll add that note to the astro repo docs once you give the word.

that it may be, but also may not be. and also that sometimes the fix is in editing the org-mode source file. (but that the solution is never editing the mdx directly, since that gets automatically generated and overwritten.)

At this point, I switched from Codex to Claude Code, because Claude Code is better at writing clear English than is Codex, which is terse and token-efficient to the point of being cryptic and hard to parse.

I achieved the handoff from Codex to Claude Code simply by copy-pasting the Codex transcript, saying simply:

Analyze this exchange.

Analyzing this exchange, I see several key insights:

What Happened

1. Problem: Inline #+THEME: claude was being promoted to frontmatter, making the whole page render as Claude theme instead of just the inline
section
2. Your Recognition: You correctly identified this required fixing the ox-astro exporter (different repo), not the Astro codebase
3. The Meta-Lesson: You want to document this decision-making protocol in the astro-monorepo docs so future AI agents know when to fix:
- The Org source file
- The ox-astro exporter (different repo)
- The Astro templates/styles
- But never the MDX directly (auto-generated)

The Key Insight to Document

The protocol isn't just "sometimes you need to edit ox-astro" - it's a three-way decision tree:

When fixing content/frontmatter/markdown issues:
1. Fix the Org source if the content intent is wrong
2. Fix ox-astro if the exporter logic is wrong (how Org→MDX conversion works)
3. Fix Astro only for layout/visual/styling needs
4. NEVER fix MDX - it's generated and will be overwritten

Where to Document This

This should go in the astro-monorepo's CLAUDE.md file (which you mentioned gets symlinked to AGENTS.md) under the existing "Core Philosophy: Fix the Code, Not the Content" section, since it's related but more specific.

The principle is: Fix at the right layer - don't patch symptoms in Astro when the root cause is in the content pipeline (Org source or ox-astro exporter).

Should I add this protocol to CLAUDE.md?

yes, exactly, but with one additional nuance: we can fix org source even if the content intent isn’t “wrong”.

Example: if i say something like “let’s make it so that when it exports, it adds a theme marker”, we don’t necessarily have to do a complex code
change to ox-astro to make it automatically detect and add a theme marker. that’s because the best solution may be to keep it simple and say “Jay, the easiest way to do that is for you to add a theme marker line directly to the org-mode source file.”

Does that make sense?

Perfect sense! The decision tree is actually:

When fixing content/frontmatter/markdown/theme issues:

1. Fix the Org source if:
- The content intent is wrong, OR
- Adding/changing something in Org is simpler than coding logic in ox-astro
- Example: "Add this theme marker" → Just tell Jay to add #+THEME: claude to the Org file, don't build auto-detection logic
2. Fix ox-astro if:
- The exporter logic is wrong (how it converts Org→MDX)
- Example: THEME keywords should only promote to frontmatter if before first heading
3. Fix Astro only for:
- Layout, visual styling, or presentation needs
- Not to compensate for exporter or content issues
4. NEVER fix MDX - it's auto-generated and will be overwritten

The key insight: Prefer simple manual edits to Org source over complex automation in ox-astro, unless the automation solves a recurring pattern.

Should I add this nuanced protocol to CLAUDE.md?

yes. but also, sometimes we need to modify ox-astro even if the exporter logic isn’t “wrong”, simply because we want to add a new feature.

also, we should feel free to fix Astro, that’s fine. it’s not like a last resort, it’s fine to change it if that’s the best way.

⏺ Perfect! So the protocol is actually about choosing the right layer for the fix, not a hierarchy of last resorts:

When fixing content/frontmatter/markdown/theme issues:

1. Fix the Org source if:
- The content intent is wrong, OR
- Adding/changing something in Org is simpler than coding logic in ox-astro
- Example: "Add this theme marker" → Just tell Jay to add #+THEME: claude to the Org file
2. Fix ox-astro if:
- The exporter logic is wrong (incorrect Org→MDX conversion), OR
- You're adding a new feature to the export pipeline
- Example: THEME keywords should only promote to frontmatter if before first heading
3. Fix Astro for:
- Layout, visual styling, or presentation needs
- Template logic, component behavior
- This is totally fine when it's the right layer for the change
4. NEVER fix MDX - it's auto-generated and will be overwritten

The principle: Fix at the right layer in the pipeline (Org source → ox-astro exporter → Astro renderer), and prefer simple solutions over complex automation.

Let me add this to CLAUDE.md:

19 + * Cross-Repo Fix Protocol (Org → MDX → Astro)
20 + If rendered frontmatter/theme/markup is wrong, fix the Org source or the Org → MDX exporter (ox-astro), even though it lives in a different repo.
21 + Only change Astro templates/styles for layout/visual needs - not to compensate for exporter or content issues.
22 + ** The Pipeline

45 more lines

Let me add this to CLAUDE.md:

19 + * Cross-Repo Fix Protocol (Org → MDX → Astro)
20 + If rendered frontmatter/theme/markup is wrong, fix the Org source or the Org → MDX exporter (ox-astro), even though it lives in a different repo.
21 + Only change Astro templates/styles for layout/visual needs - not to compensate for exporter or content issues.
22 + ** The Pipeline
23 + Content flows through three layers:
24 + 1. /Org source files/ (=.org=) - Jay's authoring layer
25 + 2. /ox-astro exporter/ (different repo) - Converts Org → MDX
26 + 3. /Astro renderer/ (this repo) - Renders MDX → HTML
27 +
28 + ** Decision Protocol: Which Layer to Fix?
29 +
30 + When fixing content/frontmatter/markdown/theme issues, choose the right layer:
31 +
32 + *** Fix the Org Source When:
33 + - The content intent is wrong, OR
34 + - Adding/changing something in Org is simpler than coding logic in ox-astro
35 + - /Example/: "Add this theme marker" → Just tell Jay to add =#+THEME: claude= to the Org file, don't build auto-detection logic
36 + - /Principle/: Prefer simple manual edits over complex automation (unless solving a recurring pattern)
37 +
38 + *** Fix ox-astro (Different Repo) When:
39 + - The exporter logic is wrong (incorrect Org→MDX conversion), OR
40 + - You're adding a new feature to the export pipeline
41 + - /Example/: THEME keywords should only promote to frontmatter if before first heading
42 + - /Location/: =/Users/jay/Dropbox/github/ox-astro/=
43 +
44 + *** Fix Astro (This Repo) When:
45 + - Layout, visual styling, or presentation needs
46 + - Template logic, component behavior
47 + - This is totally fine when it's the right layer for the change
48 + - /Not for/: Compensating for exporter or content issues (fix those upstream)
49 +
50 + *** NEVER Fix MDX Directly:
51 + - MDX files are auto-generated by ox-astro
52 + - Any direct edits will be overwritten on next export
53 + - Always fix the Org source or the exporter instead
54 +
55 + ** Key Principle
56 + /Fix at the right layer in the pipeline./ Don't patch symptoms downstream when the root cause is upstream. Prefer simple solutions
+ over complex automation.
57 +
58 + ** Example Decision Tree
59 + - "Theme is wrong" → Check: Is it in the Org source? If yes, fix Org. If export logic is wrong, fix ox-astro.
60 + - "Need new export feature" → Fix ox-astro
61 + - "Styling looks bad" → Fix Astro CSS/templates
62 + - "MDX has wrong syntax" → NEVER fix MDX; fix Org source or ox-astro
63 +
64 This applies to parsers, formatters, converters, and any code that processes user content.
65
66 Add insights to `docs/codebase-wisdom.org`. Only log things that are /non-obvious/, /repeatable/, and /action-saving/.

Perfect! I've documented the Cross-Repo Fix Protocol in CLAUDE.md with:

1. The Pipeline - Clear visualization of the three layers
2. Decision Protocol - Specific guidance for when to fix each layer:
- Org source (prefer simple edits over automation)
- ox-astro (wrong logic or new features)
- Astro (styling/templates - totally fine!)
- MDX (NEVER)
3. Key Principle - Fix at the right layer, don't patch symptoms downstream
4. Example Decision Tree - Concrete scenarios

This captures the nuanced insight that you want simple manual Org edits when appropriate, and that fixing Astro is perfectly fine when it's the
right layer - not a last resort. The protocol helps future AI agents make the same smart decision you made when recognizing the theme issue
needed to be fixed in ox-astro.

How to Actually Make Your AI Smarter

How to Make Your AI Smarter

Related Posts

How to Generate a ChatGPT Wrapped

The Myth of Sycophancy

I Finally Asked My Dad the Question I’d Been Avoiding for Years

Equipment for Living