The AI Career Inversion: Why Junior Engineers Must Start Thinking Like Architects

Written 20 hours, 48 minutes ago

Today, I open-source Scenario System Design (SSD), a 3 step system design that allows an engineer to design a sustainable system architecture without spending a mind-boggling amount of time releasing the feature.

This all started during my days working in startups. I would be filled with ambition, designing a system so grand and powerful, able to cater to traffic in the millions. You see, startup founders would, naturally, fill me with dreams of a wildly successful business. People will flock to our business really fast once we launch the product. I must be ready for the riches to come.

What turned out was a product that never saw daylight for more than a year, and the startup ran out of money. Even if I wanted to launch the product, the product was so incomplete that even if I launched it, it barely even accomplish its primary function. It doesn't serve anybody.

And so, the startups fail. I wouldn't say that it's entirely my fault or I'm blameless. But this immature naive gullible me got a little too ambitious and tried to build a cathedral on pennies.

Then came jobs that required me to deliver results in record time.

• Marketing is launching a promotion next week.
• Operations is managing a new product line this Friday
• There's a bug customers are facing. Fix it now!

Naturally, I would find the quickest and shortest way to accomplish the task. With such time pressure, there simply isn't any leeway to think long term. I was caught up in the moment, despite a tiny voice bugging, that my decisions today would haunt me sooner or later.

And true enough, as the technical debt pile on and on, it eventually led to a system so fragile that changing anything could easily break the application. It led to a point where I spent more time bugfixing than delivering features. I would going to business leaders telling them how the system needed a revamp. The leaders would be so frustrated at a system that frustrates the customers. I would be constantly fixing one bug after another while the competitors are delivering feature after feature. And you think that maybe if I adopted DevOps, it will help to speed up delivery? No No No... I simply incurred technical debt faster, launched faulty apps faster, and frustrated more audiences faster.

You see, DevOps is good at streamlining development to production, but it doesn't make up for poor design. If the application you designed cannot work without redesigning the database schema or refactoring large swathes code, no amount of DevOps will make it work. You must spend this manhours to revamp the application. In a fast-moving business landscape that requires us to be nimble, the heavy footsteps of a revamp ain't getting any greenlight.

So after all the frustrations I've gone through, I got thinking, what if we could prevent this situation in the first place? I mustn't a design that incurs all the technical debt, but I ain't going to make the mistake of designing a grand system that never sees the light of the day. It's through these many trial and error across many projects that I've honed this 3 step process I call Scenario System Design.

It starts with coming up with every conceivable scenario for the feature you are going to build, very similar to the startup days where I would go about building any sort of scenarios I was so hyped up about. The second step would involves designing a grand system that supports the scenarios you've conjured up in your brainstorming session. But the third and final step is what makes thing practical, the implementation step, which strips out anything in the grand design that you don't need to build now.

And thus, I teach SSD to my fellow peers in the companies I work for. It is through my interaction with my peers that I noticed not many engineers practise SSD. After all, the entire industry has pretty much standardise on 2 methodologies that doesn't encourage building in such manner, Waterfall and Agile methodology. While both of these are good on its own right, it doesn't solve the fundamental problem, that you need to build a system designed for tomorrow, but built for today's environment.

But the most important factor to why I'm sharing this out in the open today, is due to AI.

Before AI was doing much of engineer's day-to-day task, juniors in the company would spend much of their time writing codes, honing those skills towards writing efficient and maintainable codes. A good design wouldn't be at the top of their mind. A senior would typically step in to design the system and hand the bluepriont down to the young padawans to implement. THe seniors would also review the codes to see that they have written an efficient and maintainable codes, spotting silly mistakes like N+1 or major security lapse.

But AI has changed the entire engineering career ladder. AI is generating far more efficient and AI-maintained codes than our junior engineers are doing. That meant scope of work juniors handled is no longer relevant in the ladder. What're juniors suppose to do now? What do engineering managers do with the juniors in their team? Why would businesses bother hiring juniors? These were the questions I've been asking myself and my team lately. It makes no sense that their role is simply to prompt AI to perform work. Anybody can prompt now.

As such, I have started introducing SSD to the juniors in my team. What used to be in the domain of senior engineers and tech leads, now I bring my years of knowledge and experience down to juniors. And for the foreseeable future, this is what juniors will be trained upon. What will be the contributions of seniors and tech leads then? That I do not know. But one thing for sure, every engineering role has changed for good.

This reminds of me an old case study highlighted by MIT economist David Autor about bank tellers and Automated Teller Machines (ATM). Before the advent of ATMs, tellers were the frontliners handling every banking needs, from simple deposits and withdrawals to bank loan underwritings, they handle it all. But when ATMs started proliferating, every banker started worrying about their jobs. If the machines are taking over their job, what will the bankers do for a living? Many predicted the loss of many banking jobs, and sure enough, many roles in the banks were eliminated. But instead of the banking industry shrinking, it actually expanded. The bankers instead went on to take on much more complicated work, such as retirement planning, wealth planning, and so on. While the ATMs handled the bulk of mundane work, the tellers went on to handle higher-value tasks, which in turn increased the demand for banking services and the banks increased hiring to meet the demand.

Citation:

• Why Are There Still So Many Jobs? The History and Future of Workplace Automation
• Will automation take away all our jobs?

Are we seeing the same thing in engineering? I suspect so. And so, I give you SSD, to help transition teams towards higher-value contributions to their organisations they work for.