Technical Leaders Must Build: Why AI Leadership Requires Hands on Keyboards

I have a strong opinion that I want to put on the record: if you are a technical leader in 2025 and you are not personally building with AI tools, you are failing your team.

Not evaluating. Not reading reports. Not attending vendor demos. Building. Hands on keyboard, prompts in terminal, shipping code with AI assistance. There is no substitute for direct experience, and the gap between leaders who build and leaders who delegate their understanding is becoming a chasm.

The Knowledge Gap is Real

I talk to engineering directors and VPs regularly. The pattern is consistent. They know AI is important. They have approved budgets for Copilot licenses. They have read the McKinsey reports about productivity gains. They can speak fluently about AI strategy in board meetings.

But ask them to describe what it actually feels like to pair with Claude on a complex refactoring task, or how agent-based workflows differ from chat-based assistance, or what breaks when you try to use AI for system design versus implementation, and the conversation gets vague quickly.

This is not a criticism of intelligence or intent. These are smart, capable leaders. The problem is structural: they have outsourced their understanding of AI's capabilities and limitations to their reports, to vendor marketing, and to industry analysts. They are making decisions about tooling, process, and team structure based on secondhand information.

In any other domain of engineering leadership, we would recognize this as a problem. A VP of Engineering who has never written a line of code cannot effectively evaluate architectural decisions. A CTO who has never operated a production system cannot meaningfully assess reliability engineering practices. The same principle applies to AI: if you have not used it deeply, your judgment about it is unreliable.

What Building Actually Teaches You

When I started building Loki Mode, I learned things about AI agent systems that no amount of reading could have taught me.

I learned that LLMs are remarkably good at certain types of code generation but consistently struggle with others. Boilerplate code, CRUD operations, standard patterns: these are reliably strong. Complex state management, subtle concurrency issues, performance-sensitive hot paths: these require significant human guidance and verification.

I learned that context management is the real engineering challenge. The model's context window is a hard constraint, and how you manage it determines whether your AI-assisted workflow produces good results or garbage. This is not obvious from demos or documentation. You have to hit the wall yourself to understand the shape of the constraint.

I learned that verification is more important than generation. Getting an AI to write code is easy. Knowing whether that code is correct, performant, secure, and maintainable requires engineering judgment that the AI itself cannot reliably provide. Leaders who understand this will build different processes than leaders who think AI is primarily a code generation tool.

I learned that the productivity gains are real but unevenly distributed. Some tasks see 5x or 10x improvements. Others see marginal gains or even negative productivity when you factor in the time spent debugging AI-generated code. Understanding this distribution is critical for setting realistic expectations and designing effective workflows.

None of this is in the vendor pitch deck.

The Delegation Trap

Here is how the delegation trap works. A VP decides AI is strategically important. They assign a senior engineer or a small team to "evaluate AI tools and recommend an approach." The team spends a few weeks experimenting, writes a report, and presents findings. The VP approves a rollout plan based on the report.

The problem is not the process; it is the information loss. The engineer who spent weeks building with AI tools developed deep intuitions about what works and what does not. Those intuitions are difficult to communicate in a report. They are the kind of tacit knowledge that comes from repeated interaction: the subtle patterns of when AI suggestions are trustworthy versus when they need scrutiny, the types of prompts that produce good results versus those that lead the model astray.

The VP receives a distilled version of this knowledge, stripped of nuance. They then make organizational decisions based on that distilled version. The resulting strategy may be directionally correct but will almost certainly miss important details that would change specific decisions about workflow design, tool selection, and team structure.

This is why leaders must build themselves. Not to replace the evaluation team, but to develop their own intuitions that allow them to ask better questions, challenge assumptions, and make more informed tradeoffs.

What I Recommend

If you are a technical leader who has been observing AI from the sidelines, here is what I recommend.

Start with a real project. Not a toy example. Not a tutorial exercise. Pick a real piece of work that you would normally do yourself, something with enough complexity to expose the edges of what AI can handle. A refactoring task, a new API endpoint, a testing infrastructure improvement. Something that matters.

Use the tools directly. Install Claude Code, Cursor, Copilot, whatever your team is using or evaluating. Use the CLI tools, not just the IDE integrations. Understand how the tools work at the interface level, what they expose, what they hide, and how they handle different types of tasks.

Push past the initial impressions. The first hour with AI tools is magical. Everything seems to work. The second day is where reality sets in: hallucinations, context limits, subtle bugs in generated code, suggestions that are syntactically correct but architecturally wrong. You need to experience both the magic and the friction to have a complete picture.

Try agent-based workflows. If your team is evaluating autonomous agents or multi-agent systems, you need to understand what that means in practice. The difference between chat-based AI assistance and agent-based autonomous execution is not incremental; it is a different paradigm. Run an agent on a task and observe the entire process, not just the final output.

Keep a log of what works and what does not. Document the specific tasks where AI was genuinely helpful, the tasks where it was neutral, and the tasks where it slowed you down. This empirical data will be more valuable than any analyst report because it is grounded in your specific context and codebase.

The Leadership Advantage

Leaders who build with AI develop advantages that are difficult to replicate through delegation.

They can set realistic expectations. When a team estimates that AI will cut development time by 40%, a leader who has personally experienced the productivity distribution can ask informed questions. "Which tasks are you assuming the 40% applies to? What is your plan for the tasks where AI does not help?"

They can design better processes. Understanding the verification problem, the context management challenge, and the uneven productivity distribution leads to different process designs than assuming AI is a uniform productivity multiplier.

They can evaluate tools more effectively. When you have used multiple AI tools for real work, you develop preferences and opinions that are grounded in experience. You can distinguish between tools that demo well and tools that work well in practice. These are not always the same.

They can mentor their teams. Engineers learning to work with AI tools benefit from guidance by someone who has navigated the same learning curve. A leader who can say, "I hit that same problem, here is what I learned," is more effective than one who can only point to documentation.

They can make better hiring decisions. As AI reshapes the engineering role, understanding what changes and what remains constant is critical for hiring. Leaders who build with AI understand which skills become more valuable (system design, verification, prompt engineering) and which become less differentiating (boilerplate implementation, standard pattern application).

This Is Not Optional

I want to be direct about this. In 2025, building with AI tools is not an optional enrichment activity for technical leaders. It is a core competency.

The leaders who will navigate the AI transition effectively are the ones who understand the technology from the inside, who have developed their own intuitions through direct experience, who can make informed tradeoffs because they have personally encountered the constraints.

The leaders who rely on secondhand information will make decisions that are directionally correct but specifically wrong. They will over-invest in areas where AI underperforms, under-invest in areas where AI excels, and design processes that do not account for the technology's real strengths and limitations.

The gap between these two groups is widening. Every month that passes without hands-on experience is a month of falling further behind.

Pick up the tools. Start building. Your team needs you to understand this technology firsthand, not from a summary slide.