🇼🇸 Talofa! O lo'u igoa o Tausani Ah Chong, e sau mai le nu'u o Puipa'a ma Avao

📍 Central Tāmaki Makaurau, Aotearoa

💼 Intermediate Software Developer @ Vector ⚡️

👨‍👩‍👧‍👦 Proud dad of 2 kids

📝 Post #6 - Posting every week until my birthday in April 🎂

Follow me on Instagram! @tausani.376

GitHub: https://github.com/tausani-ah-chong

TLDR;

My company uses M365 Copilot, and for good reason, enterprise guardrails matter.

But it has no API, no terminal access, no agentic tooling.

So I built my own coding agent around it. And it works 🎉

Here’s what I learned.

The constraint that I just couldn't shake

No api. No cli. No agentic tooling.

That's the reality of working with M365 Copilot.

The only access you have is through the web or native app.

Which means you chat to it, if you like something then you copy and paste it to your IDE.

My cmd+c, cmd+v has never been faster 😅 up until last week though 👀

The contrast after playing with tools like Claude Code or Codex is crazy different.

I wanted more out of M365 Copilot since the above tools aren't approved to use at work.

Ive been trying to find a way to do something similar that can run in your terminal.

I finally cracked it.

I don't even remember when the idea popped in my head but I went and tried it and yup this was "the last ingredient I needed to make this work"

I've pretty much closed the loop im making M365 Copilot agentic.

Meaning I run my application from my terminal, set a task, and away it goes reading and editing files until the task is complete.

So yeah, I'm building a coding agent from scratch. And it's been the most fun with constant learning.

In this post, Im going to share with you a few learnings I think every software engineer should know about.

I've already spent my small career learning to build apps, now it's time to learn to build the thing that builds the thing.

Here it is in action:

Non agentic development flow

Let's take a look at what the current flow for any developer using M365 Copilot looks like:

As you can see, it is very simple and linear.

User -> M365 Copilot -> copy + paste -> IDE

Bruz you could even swap out M365 Copilot for Stack Overflow 🫣 See what I mean - it feels old school already.

But hey it works, but we all know it can be a lot better. Read the room.

Agentic development flow

Below is where we introduce an agentic loop & "the last ingredient I needed to make this work". This is the very same foundation of tools like Claude Code & Codex:

So a couple new terms to get used to if you are not already familiar:

Agent loop

Nothing fancy, it's just a while-loop with an AI assistant.

You start with a task, and the loop ends when the task is complete.

Tool calling

This part is 🔥

You tell the AI assistant you have some tools they can use during the loop.

Things like reading, editing, searching in files & running bash commands.

All it has to do is respond by calling the tool of choice. The loop returns back to the AI assistant the output of these tools.

From here an automated copy + paste loop is born.

You will notice if you use Claude on the web you can pick the built-in tool to browse the web.

Human-on-the-loop

There's a reason they called something YOLO mode for coding agents. To give full confidence and permission to control your machine 😬

Or you can sit in the driver's seat and approve tool calls, steering the ship.

There are tradeoffs but it's better to lean on smart defaults.

The loop will pause until human approval of tool calls.

Example of my read_file tool:

The build

Enterprise barriers? Hold my beer 🍺 (Shoutout to my toko Tino for giving me this feedback on this project)

The idea was the easy part.

There was already infrastructure to learn from in terms of foundations of agent loops, TUI and tool calling.

The hardest part was actually parsing the responses from Copilot.

Now please bear in mind this agent is still a young puppy, so it still doesn't work all the time for the range of responses the model returns to us. It is a work in progress.

The part I'm still learning about is the relationship between a model like Anthropic's Opus and agentic tooling like Claude Code.

For example, built into the api to call Claude models is a parameter for a list of tools:

But because we are using "the last ingredient to make this work" we don't have an api to call.

The limitation is that the only option we have is to send our list of tools as part of our system prompt.

My guess is with the above api calls with tools baked into the api, there is some other mechanism on the other side of the api to manage the way the responses get given back as tool calls.

Models like Opus and Sonnet are trained to return tool calls as first class citizens, whereas with my approach without an api and just purely with a system prompt, all we can do is hope the model responds accordingly.

This is where non determinism could work against us.

Not because the code or system prompt was wrong but purely because my tool calls aren't first class citizens.

Read my previous post about determinism vs non determinism

You can see the errors in the screenshots below.

There's been a few times where we can't parse properly and our tool calls don't register even though the response we give to the user looks fine.

It comes down to what is in the full response. Sometimes the tool call is buried within the response.

We'll get there with more iterations and the more I tweak the system prompt maybe? I have a few ideas.

"the last ingredient to make this work"

There's slang in NZ that gets thrown around whenever you feel let down or led on by someone. You would call someone or something "Build ups".

Where they've hyped something up so much that the hype was more entertaining than the actual reveal 💀 You don't want to be build ups, not even your dad.

So I'm hoping this last ingredient isn't build ups.

Because I know you've read this far, and I thank you, you're the real deal.

So here goes..

My last ingredient...

...

The ingredient is Playwright

...

All I needed was to automate copying and pasting requests and responses between the loop and M365 Copilot.

With Playwright you can spin up a browser and interact with a web page using code. Simple right?

So there we have it, some of the best ideas come from free rein and no restraints..but lets not forget about even greater ideas that come from restraints.

Enterprise barriers are there for good reason, and I fully agree that enterprises with big contracts and influence should be very critical of any new tech and tooling.

Last question for you:

Was my reveal build ups? 😆 let me know by commenting on my linkedIn post, since you made it this far 🫰

See you in the next one!

Thanks for reading! If you enjoyed this, follow me on Instagram @tausani.376 and connect with me on LinkedIn Tausani Ah Chong.

I'm posting every week until my birthday in April - tune in for the next one!

🇼🇸 Talofa! O lo'u igoa o Tausani Ah Chong, e sau mai le nu'u o Puipa'a ma Avao

📍 Central Tāmaki Makaurau, Aotearoa

💼 Intermediate Software Developer @ Vector ⚡️

👨‍👩‍👧‍👦 Proud dad of 2 kids

📝 Post #4 - Posting every week until my birthday in April 🎂

Follow me on Instagram! @tausani.376

GitHub: https://github.com/tausani-ah-chong

The model just wants to use tools

That's the part that stuck out for me from Boris (Creator of Claude Code) from his latest chat with Gergely Orosz on The Pragmatic Engineer.

This came up after he spoke about what tools or skills you will need to succeed in this new era of agents/AI.

He basically said it's more useful now to understand one level deeper than the current abstraction that you are working at. For example in the past if you were a frontend dev working with React, one level deeper would be understanding how it works under the hood to get the most gains.

But what does that look like now? Is it frameworks like Next.js or TanStack? Or is it literally understanding how AI models work? - Since the current abstraction is prompting agents

To which Boris decided to study what is going on under the hood for Claude Code, which is the model.

The finding - "The model just wants to use tools" - means don't just put it in a box and let it run. What you should be doing is setting it free and giving it tools to use.

Which led me to thinking ok, let's put this into practice - what tool should I build…

Introducing… FAALUPEGA CLI 🎉

So you're probably on one of 2 sides. One side you're probably like what is that? And the other side you're like 😬 ooh um are you tackling that?

What is Faalupega? It is probably one of the most important records to Samoa. These are records of Samoan chiefly honourifics grouped by village called O Le Tusi Faalupega o Samoa or just a Faalupega. Before any records were written (before Samoan was a written language) it was all oral and passed down through generations and is still practiced to this day by master Orators, or we call Tulafale.

There are over 150 villages across Samoa and each with their set of Matai chiefly titles - so we need an easy way to search across villages and matai title names.

And just like the technology upgrade from oral to written to printed books, it is definitely time to adapt to the new era in AI. So it's important that this should be able to be easily used by both AI agents and other technical folks.

There were a few other ideas for tools that came to mind but thought I would go with the most unique and one that will actually help myself, and I'm sure others that are on the same journey as me learning about Faasamoa and Faamatai - the ways of Samoa and chiefly honourifics.

I figured it would be a great research tool and a foundational layer for both technical folks or AI agents to use. We'll get to a nice UI soon - or better yet you could build it using this tool 😉

Also why a CLI? Why not just build a website first? Because anyone can build a website or app, what's missing is the foundation and underlying platform. This CLI will be one of the first building blocks.

Tradeoffs

I've never built a CLI tool before, and instead of telling you what prompts I used, here are a couple tradeoffs I had to think about.

1. Language

It was basically TypeScript/Node vs Rust.

For example OpenAI's Codex was first built with TypeScript/Node but then rewritten in Rust.

TypeScript:

• Faster to prototype? (I don't think this is a thing anymore with AI coding agents imho)
• I am already used to reading and understanding the Node & npm world

Rust:

• Yeah definitely better performance so they say lol - I've never built anything with Rust
• Apparently builds all the way down to a single binary - which means you only need to download 1 thing
  • As opposed to TypeScript, the user also needs to download Node - although you can package it together now with Bun

This decision would also influence the next tradeoff…

2. Distribution

I come from a web dev background where the only distribution is the browser, so I never needed to think about users needing to access apps I built. I'm just glad I fell into tech after the Internet Explorer era 🤏

Now with a CLI I had to think about different OS's - your Macs, Linux or Windows users. It seems like the way to go is to support all.

3. API vs Local Storage

I don't mean local storage from the browser context, but instead in-memory.

Sure I could create an API and store data in a database - this way even more users will be able to access the data.

The downside to in-memory is the size of bundle and also staleness of data. But the upside is I can continue to keep building on the small features and data I currently have.

The CLI in Action

I wanted to keep the CLI initial features very basic - just a couple text match searches, via village or matai title name. This is the main product: you would usually look up a village to study, or partial match on matai names across villages.

You will also notice a Version header at the top of results to let the user know what data source they a looking at

Search by village:

$ faalupega village puipaa

┌──────────────┐
│ Version 1930 │
└──────────────┘
════════════════════════════
  PUIPAʻA - Faleata, Upolu
════════════════════════════

Tulouna lau tofa ʻUlu ma lo outou aiga o Seiulialii ma Laufou

MALAE-FONO:
  Lepea - Fono o le manino (filemu)
  Vaitagutu - Fono o le ʻaʻava (taua)

MAOTA O ALII:
  Mataiʻa - Vaiala
  Faumuinā, Letele, Mataʻafa - Lepea
  Seiuli - Vaiusu
  Pepe, Fanene - Falefasa

O IGOA-IPU A ALII:
  Mataiʻa - Tolotūpō
  Faumuinā:
    1. Fai ʻava le ita
    2. Talitigā
    3. Numia ma Tumua
  Seiuli - Seufagafaga
  Le Mataʻafa - Pago talitali le ipu a e taute le Mataʻafa
  Galumalemana - Lagofaatasi
  Letele - Fotualagoataata
  Pepe - Taumailelei
  Fanene - E tapa fua
  Manuʻa - Alo talitali le ipu a e taute le Manuʻalesā
  Latafale - Tamaalii faaea Faleupolu
  Sāvali - Ulugia ma Faleʻafa

SAʻOTAMAʻITAʻI:
  Faumuinā - Letelesā
  Mataiʻa - Latafale
  Seiuli - Tooā
  Pepe, Fanene - Latafale (Safune)
  Leitulua - Muniao
  Une - Tonuu
  Leleua - Ufagafā
  Ale - Tauaiupolu
  ʻUlu - Alofasaupo

Search by matai title name:

$ faalupega matai ale

┌──────────────┐
│ Version 1930 │
└──────────────┘
════════════════════════════
  PUIPAʻA - Faleata, Upolu
════════════════════════════

  MAOTA O ALII:
    Pepe, Fanene - Falefasa
  O IGOA-IPU A ALII:
    Galumalemana - Lagofaatasi
    Manuʻa - Alo talitali le ipu a e taute le Manuʻalesā
    Latafale - Tamaalii faaea Faleupolu
    Sāvali - Ulugia ma Faleʻafa
  SAʻOTAMAʻITAʻI:
    Mataiʻa - Latafale
    Pepe, Fanene - Latafale (Safune)
    Ale - Tauaiupolu

═══════════════════════════
  TOAMUA - Faleata, Upolu
═══════════════════════════

  TULOU:
    Tulouna lau tofa Ale ma Inaʻilau na oo i lo outou aiga,
    o Tupalaimuli ma Tauaiupolu, ma le Aoaniu ma le Momoti

--help command:

$ faalupega --help

Usage: faalupega [options] [command]

Samoan Faalupega Lookup Tool

Faalupega is the traditional record of matai (chief) titles for each
village in Samoa. It documents the sacred hierarchy of titles, their
connections, ceremonial meeting grounds (malae-fono), houses of chiefs
(maota o alii), and high chief names (igoa-ipu a alii).

This tool provides a searchable reference for these records.
All searches are partial and case-insensitive. Samoan diacritics
(macrons: a, e, i, o, u and glottal stops: ʻ) are optional in queries
but preserved in output.

Usage examples:
  $ faalupega village Puipaa          Search by village name
  $ faalupega nuu Puipaa             Same search using Samoan alias
  $ faalupega village pui             Partial match
  $ faalupega matai Seiuli            Search by matai title
  $ faalupega suafa Fanene            Same search using Samoan alias
  $ faalupega village Puipaa --json   JSON output for agents

Options:
  -V, --version                 output the version number
  --data-version <version>      Faalupega version year to query (e.g. 1930) (default: "1930")
  -h, --help                    Display help for command

Commands:
  village|nuu [options] <name>  Look up a village's faalupega by name. (Samoan: nuu / nu'u)
                                Searches are partial and case-insensitive. Diacritics (macrons, glottal stops) are optional.
                                Examples: faalupega village Puipaa, faalupega nuu Puipaa, faalupega village pui
  matai|suafa [options] <name>  Search for a matai (chief) title and find which villages it belongs to. (Samoan: suafa)
                                Searches across all sections: Tulou, Malae-Fono, Maota o Alii,
                                O Igoa-Ipu a Alii, and Saʻotamaʻitaʻi.
                                Partial and case-insensitive. Diacritics optional.
                                Examples: faalupega matai Seiuli, faalupega suafa Fanene, faalupega matai ulu
  setup                         Interactive setup for faalupega.
                                Displays a welcome banner and lets you choose the default version.
                                Your choice is saved to ~/.faalupega/config.json.
  help [command]                display help for command

Versions and the Source of Truth

Naturally there are parts about the Faalupega in general that are a bit controversial and people take pride (and they should) in the hierarchy and order of the honourifics based on who is most important. There is not one single source of truth. Since the records have been documented there are many versions, and also naturally each village or family may record their own separate Faalupega records.

So this was the next challenge and I hope that in the form of versions, much like how there are many English translations for the Bible, we would need to enforce that in this tool. A user should be able to toggle or search between versions.

There is a command that should be run when you first download it called setup here you get an interactive way to select you're default data source

Setup command:

$ faalupega setup

Output:

And in future I would hope they too can upload their own records alongside the publicly available ones - damn we could make a marketplace 🔥 but the main point is this tool would provide the platform to research regardless of the "correctness" - as that could get me cancelled aye 😬

What's Next

So yeah go check it out here: Faalupega CLI on GitHub - it's all open source. I decided to go with TypeScript/Node for now as we build up features and add more villages to the data set. And may port to Rust eventually.

Next steps is to add more villages! I've got 3 down, 149 to go. I also am in week 3 of my 10 week Faamatai course - getting this up and running will help myself with my homework and in future hopefully anyone else wanting to learn too.

Thanks for reading! If you enjoyed this, follow me on Instagram @tausani.376 and connect with me on LinkedIn Tausani Ah Chong.

I'm posting every week until my birthday in April - tune in for the next one!

🇼🇸 Talofa! O lo'u igoa o Tausani Ah Chong, e sau mai le nu'u o Puipa'a ma Avao

📍 Central Tāmaki Makaurau, Aotearoa

💼 Intermediate Software Developer @ Vector ⚡️

👨‍👩‍👧‍👦 Proud dad of 2 kids

📝 Post #3 - Posting every week until my birthday in April 🎂

Follow me on Instagram! @tausani.376

GitHub: https://github.com/tausani-ah-chong

Ok I kind of lied to you in my previous post and I apologise

So I thought you know writing these posts would be a breeze now because there are many AI tools to help you write, (even before we had grammarly)

Ok so here we go,

my previous post was edited and some parts written by AI

whoopty doo right? Everything is written or thrown over to AI to improvise these days, but should everything?

AI, Writing, and What We're Losing

In a world now where we have flooded the internet with AI generated content, we actually lose the humanness. Now yes AI is great for repetitive tasks, and automation. But for writing?

I think this might be one of the few last frontiers, one of the last things we have, to be as genuine as we can online (apart from things like music and livestreamed content).

I can honestly say AI didn't write or help edit this for me. You are reading my thoughts directly from me, as raw as they are, you can't get much more human and genuine than that.

A Story from Outside My Day Job

A similar thing happened in a role outside of my day job. I want to share a story of mine of when I decided to put my hand up to be Secretary for my kids A'oga Amata (Samoan preschool) committee. And yes it does get more technical later down the post.

I previously had been on as a parent representative, and the former secretary was leaving so there was a gap. I knew taking on the role would be more involved but I just knew I had to step up. I had a few ideas and thought why not and lets use AI to help with the role and tasks

The main tasks are to schedule meetings, and take minutes during those minutes. So I thought lets get AI to help with taking minutes. At my current role for my day job as a Software Engineer we have access to those tools via Microsoft Teams and now with Slack AI tools. They kind of work ok and get the job done. But this time I need to be very careful with what to feed to AI to get back both transcripts and summaries. Privacy and confidentiality would be a big requirement.

Testing a Local AI Solution

So I thought what about using an AI tool that runs on my local machine and never talks to third party servers.

I researched and ended up finding a Whisper port, based off the O***AI (The company who should not be named iykyk) model but ported to C++ for better performance. Did a couple manual tests beforehand with random audio and thought I would try it with a real meeting. So before our meeting, I asked for permission first if I could record and feed into my local AI tool. I then dumped the 1 hour .wav file into the tool and let it do its thing.

I got to test a few different models that you could download that works with the tool and definitely found varying results, some even got stuck in loops. v3-turbo-q5 ended up giving me the best results. Some were just pretty bad imo

I then tried to get a summary from the transcript and try get some tasks out of it. But I soon realised the quality was pretty bad too and unusable, and missed some pretty important points. Not to the mention the added complexity of Samoan language used throughout the recording.

I could tell off the bat the performance and quality gains are definitely with paid models but you have to live by their rules of stealing and using your data to train etc say bye to your privacy. This exercise got me to question do we really need to use AI for everything? Should you really just offload all tasks for AI improvise?

Where the Rigor Goes

In this case I came to the conclusion - back to reference the "where does the rigor go?" from my previous post here - that it was a lot more accurate and critical if I actually took the effort to decide what and what shouldn't be taken down as minutes.

No one actually goes back to listen to the whole 1 hour meeting or want to read through a full wall of text.

Sure AI can summarise but how do I in the moment say ok at 22minutes 33 seconds take mark this task down as it's important. I may as well just use my pen and take it down right there and then. Intentional.

Putting intention to actively listen and scribe is a skill in its self, and putting effort and rigor in to making sure important actions are captured is waaaay better than letting AI improvise that. Maybe im a few prompts away from finding the perfect product for my use case - maybe I have to build this myself? Maybe you?

But to sum up, I think I made a good decision as I can still be intentional, which is where the rigor is. And yes its more effort, but without effort how can you expect to get better and grow in whatever your role is.

In a world full of generated content, I really do hope genuine content becomes higher impact and accessible. And if you feel you want more raw authentic content please leave a comment, share and follow along the journey.

Ia manuia le aso

Thanks for reading! If you enjoyed this, follow me on Instagram @tausani.376 and connect with me on LinkedIn Tausani Ah Chong.

I'm posting every week until my birthday in April - tune in for the next one!

🇼🇸 Talofa! O lo’u igoa o Tausani Ah Chong, e sau mai le nu’u o Puipa’a ma Avao

📍 Central Tāmaki Makaurau, Aotearoa

💼 Intermediate Software Developer @ Vector ⚡️

👨‍👩‍👧‍👦 Proud dad of 2 kids

📝 Post #2 - Posting every week until my birthday in April 🎂

Follow me on Instagram! @tausani.376

GitHub: https://github.com/tausani-ah-chong

Being a parent but also a software engineer learning about AI, the kagamea (clothes washing) doesn't stop, a'e sele 😆 but what I love about folding time is chucking on a podcast. That's when I had my most recent aha moment, the one that made LLMs and agents finally click for me.

The Aha Moment

The podcast was The Pragmatic Engineer (Gergely Orosz) interviewing Martin Fowler: "How AI Will Change Software Engineering."

It's been a hot topic for a minute now, with takes coming from all sides. I take most of it with a grain of salt and try to experiment and play around myself. But Martin Fowler walked through the brief history of programming languages:

Assembly → Fortran → C → Java → JavaScript → English (LLM prompts)

Then he mentioned two words I'd always overheard but never really stopped to understand: determinism vs non-determinism. I don't know why I never paused on them before, but hearing it in context, and a quick ChatGPT prompt later, I had my aha moment.

Here's the thing: determinism vs non-determinism has always been a concept in computer science, but it never really landed for me day-to-day. You wrote code, the code did what you told it, and the output was predictable. The compiler wasn't going to surprise you.

But now? Now we have non-deterministic systems, LLMs, directly writing our deterministic code. That tension didn't exist before. Every time Claude Code or Codex generates an implementation, it could produce something slightly different from the last run. Same prompt, different code. That's a new problem, and it's why this distinction suddenly matters to every engineer, not just academics.

Why You Should Care About Determinism vs Non-Determinism

Non-determinism is where LLMs shine. It's a quality we humans share: same input, and the output can vary each time. It's why ChatGPT, Claude, and other LLMs can sound so human.

But some software demands strict determinism: same input → same output, every time. That's how we get predictable, trustworthy behaviour.

When we use tools like Claude Code and Codex, we're trading some of that predictability for speed and output. And when you have critical software that must behave predictably, that trade-off becomes the biggest risk in the room.

What Levers Can We Pull?

A recent ThoughtWorks retreat, where Martin Fowler has worked for many years, brought together senior engineering practitioners from major technology companies to confront the questions that matter most as AI transforms software development.

Their full document is worth reading, but the line that piqued my interest was:

"Where does the rigor go?"
Engineering quality doesn't disappear… it migrates to specs, tests, constraints, and risk management.

I decided I wanted to tackle the tests part, specifically with Test-Driven Development (TDD).

Can TDD Increase Determinism?

This weekend I took Claude Code for a drive to see how much we can influence LLM outputs. Can TDD act as a guardrail that pushes AI-generated code toward more predictable, deterministic behaviour?

I tested four approaches, all given the same prompt:

"Please complete this task: A user registration service that validates email, hashes a password, and saves to an in-memory store - split across validator.ts, hasher.ts, userStore.ts, registrationService.ts"

Here's what happened with each.

1. Custom TDD Coding Agent

This was the most fun and gave me the most challenge and learnings, not that I manually wrote any of the code, but the rigor was still there.

The agent wraps the Claude API directly and enforces strict red-green-refactor through phase gating: it only transitions between phases based on test outcomes, limits Claude to one it() block per turn, and enforces minimum implementation. Coming from a frontend web dev background, I quickly learned you need to treat LLM API responses just like any other API call. Account for failures, non-deterministic output, and what the experience looks like when there isn't a happy path.

It got me thinking: what should the user experience be when you're getting an LLM to do TDD? Should the user be in the driver's seat approving each cycle? Or fully commit and wait for the PR? That question led me to build history and snapshots of files during each cycle, a frame-by-frame replay of the TDD loop that's invaluable for review.

Every file write is archived in sandbox/history/{runId}/iteration-{n}-{phase}-{filename}. This gives a complete frame-by-frame replay of the TDD loop

The biggest downside is the lack of full capabilities like directory traversal, things Claude Code handles natively. But it's custom, and with more time you could craft a very focused TDD-style user experience.

Source code: tdd-agent

2. Claude Code with TDD Hooks

This approach gets the best of both worlds: the power of Claude Code, plus stop gates that enforce TDD before the agent writes implementation code.

We wrote two hooks wired into Claude Code's PreToolUse and PostToolUse events that intercept all file writes. The pre-hook blocks any implementation write if a corresponding .test.ts file doesn't already exist. The post-hook runs npm test after every write. For test files it expects failure (red phase), for implementation files it blocks if tests fail and signals green if they pass.

This creates an enforcement loop that Claude Code shouldn't be allowed to bypass. No test file? No implementation. Blocking happens at the tool level, not the prompt level, making it much harder to avoid. And it works with any codebase or language; just change the test command.

The trade-off: no audit trail or cycle snapshots, and hooks verify test existence and pass/fail state, not test quality.

Below is what settings.json looks like to configure hooks

{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Write|Edit|MultiEdit",
        "hooks": [
          {
            "type": "command",
            "command": "tsx .claude/hooks/tdd-pre.ts"
          }
        ]
      }
    ],
    "PostToolUse": [
      {
        "matcher": "Write|Edit|MultiEdit",
        "hooks": [
          {
            "type": "command",
            "command": "tsx .claude/hooks/tdd-post.ts"
          }
        ]
      }
    ]
  }
}

Source code: tdd-hooks

3. Vanilla Claude Code (No TDD)

You guessed it, zero tests written. But in its defence, it stuck to what the prompt asked and wrote all the code in 8 seconds.

The untested code is still professional-looking: SCRYPT hashing, timing-safe comparison, UUID generation, email normalisation, sanitised return objects. But it's all unverifiable without tests and has no regression protection.

4. Claude Code with a TDD Prompt

This was the most interesting approach, not because it's the best, but because it looked like TDD without actually being TDD.

Tests before files. All tests pass. Clean code. And it produced 29 tests, more than double the 13 from either enforcement approach. But the full suites were written in one batch.

As I watched, the "red" was triggered by missing imports, not failing assertions. This is a subtle but important distinction. Authentic TDD red means: "I have a working implementation, and this new test exposes a behaviour it doesn't yet handle." What happened here was: "The file doesn't exist, so the entire test suite crashes at import." The implementation was then written to make the full batch of tests green in one go.

It was the most pepelo (deceptive) of the results, and the most useful illustration of why "tests exist" ≠ "TDD was followed."

Because the full suite of tests was created upfront, this was closer to Spec-Driven Development (SDD), a legit workflow, but just a different process to red-green-refactor.

So, Did We Actually Increase Determinism?

The one-it()-at-a-time constraint is everything

This is what separates TDD from SDD. The custom agent enforces it explicitly, "add exactly ONE new it()." Hooks enforce it implicitly, the post-hook runs tests after each write, so Claude learns to write one test and get it green before writing the next. The prompt-only approach gets the file ordering right but misses the granularity, and that's why it produced 29 tests to the enforcement approaches' 13. More tests, but less discipline. Claude planned the full suite as a spec, then implemented against it in one go.

Why this matters for non-determinism

The real argument for TDD with AI isn't just about discipline, it's that the one-test-at-a-time constraint forces the LLM to make minimal, testable changes. That reduces the surface area for non-deterministic drift. Instead of generating an entire implementation in one shot (where the model has maximum freedom to vary), TDD pins the model to small, verifiable steps.

Think of it as a spectrum of enforcement strength:

No tests          Tests exist (unverified process)       Tests + verified process
    |                          |                                |
 no-hooks              tdd-prompt-only                tdd-hooks / tdd-agent

Without guardrails, the LLM's implementation strategy is entirely non-deterministic. SCRYPT vs SHA256, class vs function, sync vs async are all free choices with no specification to anchor them. With prompt-only TDD, the tests specify behaviour, but the process could vary between runs. With hook or agent enforcement, every step is constrained. The failing test at each phase narrows the next implementation decision, pushing toward convergence across runs.

We also saw this play out in a concrete way: the unconstrained approaches added methods that weren't part of the original prompt. Extra helpers, convenience functions, things the model decided were useful on its own. In some ways, that's the upside of non-determinism, like a colleague who went above and beyond and took initiative. But is that what you want all the time? Probably not. That's ultimately up to whoever's in the driver's seat.

When to use what

If you're just vibe-coding a prototype, skip TDD. If you already have a clear mental model of the interface and just want it tested, prompt-only TDD is probably good enough.

But if you have critical software where mistakes are expensive, or complex domain logic where the design needs to emerge test-by-test, that's where enforced TDD pays for itself.

Claude hooks are the practical middle ground. They enforce the process mechanically without requiring developers to trust the model's self-reporting, and they scale to any project without custom infrastructure. With the downside that you're locked in to only using Anthropic models and agents. After a quick search not even Codex has this feature, it was actually a rejected PR from OpenAI.

The core finding in one sentence: prompt-only TDD gets you a tested codebase; enforced TDD gets you a codebase that was built by tests, and only the enforcement approaches can prove the difference.

The possibilities are endless if you go full custom agent, which sounds like a viable approach for companies that care about their products.

What I Took Away

I think we achieved it. The enforcement approaches (the custom agent and the hooks) demonstrably constrained the model's freedom at each step, and that's exactly what determinism is about: narrowing the space of possible outputs. Did both runs produce identical code? No. But they converged on the same structure, same coverage, same process. That's meaningful. The custom TDD agent flow could genuinely be its own product, with a proper UI and UX for reviewing each TDD cycle. I have more ideas now than I did before I started.

That's the move: start building, and let the experiments teach you.

Thanks for reading! If you enjoyed this, follow me on Instagram @tausani.376 and connect with me on LinkedIn Tausani Ah Chong.

I'm posting every week until my birthday in April - tune in for the next one!

🇼🇸 Talofa! O lo’u igoa o Tausani Ah Chong, e sau mai le nu’u o Puipa’a ma Avao

📍 Central Tāmaki Makaurau, Aotearoa

💼 Intermediate Software Developer @ Vector ⚡️

👨‍👩‍👧‍👦 Proud dad of 2 kids

Follow me on Instagram! @tausani.376

GitHub: https://github.com/tausani-ah-chong

Ok I'm nervous my managers might read this.
Because I'm finally saying what I actually think. 👀

You know that whole, "know your strengths and weaknesses" thing?
Straight up, writing about Software actually scares me.
I would hand brake and side step if I was to run it straight to writing (Imagine RunNation but for technical writing)

Take this:

Me building software at speed, writing seki code, shipping features?
On. Confidence through the roof.

Writing or getting asked about my honest thoughts and takes?
Yeah Nah.

If you've ever felt confident building but can't fully express yourself, then this might be for you.

This series of posts will be snapshots of the journey of how
a half-caste Samoan,
born in Central Auckland Aotearoa NZ,
musician (producer/DJ/radio host) turned Software Engineer at 30,
now a Dad,
recently bestowed as Matai for my family,
is getting better at putting full, honest thoughts on paper and out there to you.

My biggest strengths lie in grinding.
Teaching myself through tinkering.
Staying curious into late nights.
Sniffing out what's wrong or missing.
Being clear about what I like vs what I don't.

I learned that all from making music and transferred to software.

But when it comes to technical opinions? I freeze.

But this is part of being a great Engineer. You don't just ship. You explain why. You defend why. You question why not.
Shipping code feels safe to me.
Sharing opinions about software feels risky.

So here's the goal. 1 post a week until my birthday in April.

7 posts. 2 rules:

1. Make 7 posts.
2. Make the last one better than the first.

Let me know, does this count as 1 of 7? Or is this the prequel?

🇼🇸 Talofa! O lo’u igoa o Tausani Ah Chong, e sau mai le nu’u o Puipa’a ma Avao

📍 Central Tāmaki Makaurau, Aotearoa

💼 Intermediate Software Developer @ Vector ⚡️

👨‍👩‍👧‍👦 Proud dad of 2 kids

Follow me on Instagram! @tausani.376

GitHub: https://github.com/tausani-ah-chong

🇼🇸 Malo le soifua

Growing up in central Aukilagi our family house was in Grey Lynn. In the late 90's, I went to a full immersion (100% samoan speaking) a'oga amata (Pre school) called A'oga Fa'asamoa, located in Ponsonby — The first of its kind in NZ. Opened by the late Papali'i Dr Pita Taouma and wife Jan Taouma

Read info here!

Aaaanyways — My 2 kids currently attend the very same A'oga amata, where im also on the Commitee/Board as a Parent rep — My first question in my first Committee meeting was — "Who maintains the school website?"

Ended up finding out that it was a previous committee member Riki Apa who built it and was launched in 2004 — I saw a chance to modernise it 20 years later, as UI and UX hasn't changed since then

This below is the current website

https://www.aogafaasamoa.school.nz/

Tools for the build

So yo, off the bat I wanted to have a jam at trying out Cursor — The AI Code Editor

I also wanted to have a go at also:

• Hosting on a AWS S3 bucket (I've only ever used Vercel to deploy)
• GitHub Actions to deploy to S3

Cursor (IDE)

Before you start using Cursor, you'll need to configure what models you want to use.

You can sign up to Cursor Pro which has their own features I decided to just roll with the free features

You need to also supply you api keys

Built in chat feature

With Cursor the main feature is you don't have to open up a browser or another app to ask your questions or revise some code. You just can just stay inside your IDE to chat to the model of your choice.

You can either:

1. Chat in-line, with cmd + K or
2. Open chat panel cmd + L

In-line chat

Chat panel

I enjoyed the flow of creating with Cursor, it says it looks through your repo as context so keep that in mind if you decide to work on a repo that has personal or other information you don't want to link back to OpenAI or Anthropic.

Although even though it looked through the whole repo I still had to reply to some generated code and make sure to use tailwind to style for example, I would have expected that I didn't need to do that.

AWS S3

Pretty simple setup, but would be a lot simpler with a tool like Terraform, but for this I configured all in the console.

1. Create S3 bucket
2. Make it public
3. Enable static website hosting
4. Add bucket policy (permissions)

Create your S3 bucket

Name your S3

Its good to be quite specific

Un-check Block all public access

Check the box that you acknowledge. We will be using for static website hosting.

"AWS recommends that you turn on block all public access, unless public access is required for specific and verified use cases such as static website hosting."

In Properties tab scroll all the way down

Select Edit to configure Static website hosting

Enable Static website hosting and check Host a static website

You will need to double check the output of the static files from running a build command. It might be different per framework.

e.g. using Next

next build generates a /out dir and populates /out/index.html and /out/404.html files

You will now see a URL populate so you can view your website

Next go to Permissions tab and select Edit to configure Bucket policy

{
  "Version": "2012-10-17",
  "Id": "Policy1724460713351",
  "Statement": [
    {
      "Sid": "Stmt1724460711669",
      "Effect": "Allow",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": "arn:aws:s3:::<your S3 bucket name>/*"
    }
  ]
}

Paste in the above policy.

I ran into getting a 403 from trying to hit the url if you don't do this step.

Now you're all set to upload your code.

I decided to use GitHub actions to upload whenever I push to main

GitHub Actions

Simple steps also:

1. Generate yaml file
2. Add AWS secrets

You could just create a /<your-app>/.github/workflows/main.yml file without going through the GitHub website but I'll show anyways

Select Actions tab

Select set up a workflow yourself

All you need to do is edit the main.yml

Run your specific commands to build your app

Run command that will upload all contents from /out directory to your S3 and delete other contents beforehand

Make sure you're AWS_REGION matches for where you created your bucket

Commit your changes

Next you'll need to populate your AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY Repository secrets, so the above GitHub Action will work

You can add in Settings tab, clicking on Actions under Secrets and variables

Now you're setup so any changes merged to main will now trigger to upload your artifacts to S3

Results

So yeah thats me! Keep in mind this was just a quick weekend quest lol so time was limited.

Here are some changes I wanted to focus on for the first cut:

• Make use of full width of screen
• Modernise the font and keep them consistent (There are a few different fonts used in Before)
• Simplify the user experience
  • Much like 90's, 2000's style of making the website so accessible, to the point where there are multiple links and entry points to navigate the website. It ends up clogging up the page and actually confuses the user
  • Im hoping to keep the navigation in a single place at the top, and reduce the amount of information overload for each page

I may keep expanding on the site, but all in all the main learnings came from the journey of creating:

• First time using Cursor
• First time hosting on S3
• First time using GitHub Actions

Thats the part Im mostly happy about, and anything after is just a bonus

Make sure to check out both sites below and let me know your thoughts if you enjoyed this!

Ia fa

Before

Link to website

After

Link to website

🇼🇸 Talofa! O lo’u igoa o Tausani Ah Chong, e sau mai le nu’u o Puipa’a ma Avao

📍 Central Tāmaki Makaurau, Aotearoa

💼 Intermediate Software Developer @ Vector ⚡️

👨‍👩‍👧‍👦 Proud dad of 2 kids

Follow me on Instagram! @tausani.376

GitHub: https://github.com/tausani-ah-chong

Talofa! I’m Tausani, Junior Software Engineer @ Halter shaping the future of dairy farming 🐄

I'm a proud dad of 2 👨‍👩‍👧‍👦

Just over 18 months into coding, where I transitioned from a music production background 🎹 to software after a 15-week bootcamp @ Dev Academy Aotearoa

In my spare time (Which isn’t much with a small family!) I'm embracing the #BuildInPublic movement.

Always keen to learn and grow!

Follow me on Twitter! @tausani93

Join the Pasifika Tech Network discord using this form

GitHub: https://github.com/tausani-ah-chong

Malo le soifua!

This is my first digital garden post - Whoop whoop!

I've been holding back a bit, with my excuse being that I haven't found the right project to build or topic to choose.

Lately I've been inspired by my fellow peers in the Pasifika Tech Network discord as we share our new learnings and geek out about all things tech, with a island flavour of course. I've also been inspired by tech twitter raving about ChatGPT and posting about their projects, so I thought it was time I had a go.

Pretty much in this post, I'll go you through an overview rather than a deep dive of the process of building a Slack ChatGPT bot, and some the challenges and the learnings I gained.

• Repo: https://github.com/tausani-ah-chong/ask-chatgpt
• End result:

Agenda:

• Set up new Slack App/bot
• Bolt + Slack API for App Mentions and Slash Commands
• OpenAI/ChatGPT API for Completions
• Learnings When Using ChatGPT to Guide You
• Hosting, Security, and Privacy Concerns

Setup new Slack App/bot

To check set up a new Slack app/bot start off by following this link

Once you're in you'll need to also need to add scopes/permissions so that you can use the App mentions and slash commands:

Bolt + Slack API for App Mentions and Slash Commands

Bolt is a user-friendly framework developed by Slack that simplifies building Slack apps. Think ExpressJS but with a layer on top for Slack API methods

Now you can enable the App mentions events + create slash command by setting up your URL in 2 places

A couple things tripped me up here:

1. Make sure to use {your-hosted-URL}/slack/events
   1. I thought I could create any route but turns out it needs to end with /slack/events for both commands to work. Wasn't clear at all in the docs which took a bit of time trying to debug why I was getting errors
2. You need to verify your App URL
   1. Setting up an event for app mentions also required a specific URL for verification, and the documentation lacked clear examples of the necessary HTTP methods.
   2. You'll need a POST route that returns a challenge parameter
3. Don't use Express & Bolt at the same time
   1. I tried to use Express to create my POST route (In my defence ChatGPT imported Express as well as Bolt) i.e. I had Express routes for my POST route and Bolt for my command routes
   2. Instead use the 'ExpressReceiver' method to handle Express-like functionality without importing both Express and Bolt.

From there you can also write your command methods

Slash command:

i.e. /ask-chatgpt [question]

App mention:

i.e. @ask-chatGPT [question]

OpenAI API for Completions

Once your Slack app is configured you can now we can use the openAI API - OpenAI API docs.

One learning here:

Always read the API spec/docs!

I know it sounds cliche but because I was using ChatGPT to guide me making this I relied on it a bit too much that I didn't read the docs. I made an obvious mistake in hindsight.

ChatGPT gave me an outdated URL which took way too much time for me to debug. I guess that happens when the dataset was from 2022.

Yup blame ChatGPT not the developer aye

But hey, it was a good lesson learned. This is why you always need to get your hands dirty, and try build something from scratch.

Anyways heres the code (For the current API spec lol this may change in the future)

Learnings When Using ChatGPT to Guide You

So yeah back to blaming ChatGPT (lol im kidding) I think doing this project definitely cemented that AI isn't just ready yet, and our jobs are safe (for now)

Be aware of potential hallucinations and outdated information. Just use it as a tool. Foundational knowledge is still super important for you to make the best judgement.

Was still a fun way to build software though, definitely will carry on using ChatGPT (I think thats a no brainer) It still helped me get probably 70% of the project built.

Cool explanation about Hallucinations with AI:

https://www.youtube.com/watch?v=6_hjykO03N0

Hosting, Security, and Privacy Concerns

I only got it to work properly locally, but I did try host on Vercel but requests kept timing out. My guess is because I was using the Hobby account. But I still need to deep dive further. Maybe you could help me!

I also had to consider privacy and security concerns as I would have needed to store chat data in database on the server. So decided this was a little bit too out of scope for this project.

I just ended up using an in-memory solution to store the chat data

Conclusion

Building the Slack ChatGPT bot was a valuable learning experience, despite the time spent debugging the Slack and ChatGPT APIs. I gained insights into the Bolt framework, Slack API, and ChatGPT API, which will undoubtedly be useful for future projects. I hope that sharing my challenges and learnings can help others working on similar projects or those who are just getting started.

Feel free to check out the repo: https://github.com/tausani-ah-chong/ask-chatgpt

On a closing note

I want to highlight the amazing Pasifika Tech Network community. After sharing a screen recording of my project on the discord server, the head honcho OG said he was "nerd sniped" and built a ChatGPT bot for our discord within the same day. This is a prime example of community and inspiration from peers. I love this community and can't wait to see what we'll achieve together in the future.

🌱 Want to build your own Digital garden?

Follow this guide from Gerald Tuimalealiifano who is also part of the dream team on the Pasifika Tech Network discord

Follow me on Twitter! @tausani93

Join the Pasifika Tech Network discord using this form

GitHub: https://github.com/tausani-ah-chong