In every startup’s journey, countless decisions are made daily that have a ripple effect across all aspects of the company’s product and business.

Sometimes, these decisions lead down paths that can overcomplicate processes and require backtracking. This sort of trial-and-error is a necessary part of creating and growing a company. Learning from others’ mistakes can be a way to have more trial and less error, so I’d like to share a formative experience from starting my company.

LawIQ is a D.C. and Baltimore based tech company that combines technology and expertise to deliver analytics and answers to customers developing, operating and investing in energy infrastructure. The core of our SaaS platform is a massive data acquisition system that ingests hundreds of thousands of public regulatory and legal filings a year related to the large infrastructure projects driving America’s energy independence.

During the early days of our multi-year build out of our platform, we faced some difficult decisions. We were pulling in tons of filings in both PDF and poorly constructed text documents, so we could structure and convert them into actionable analytics for our future energy and financial industry customers — strongly believing we could help them make better data-driven business decisions.

I talked with our engineers and bugged them with design discussions day and night in our early days. We sat in a room and debated, experimented and read blogs written by developers from all over the world, including those engaged in resume driven development (RDD). At night, I kept asking myself questions like: how should we structure our backend? Should we use microservices or a monolith strategy? Is it the right time to use NoSQL? My favorite question was – and still is – how should we use the world’s most advanced text extraction technologies?

Seeking to answer that question, I made the wrong decision in the key area of whether to use monolithic or microservices. I fell victim to the sweet siren song of microservices without thinking through the end consequences down the road for a bootstrapped team.

Microservices vs. monolithic

Back in the stone age year of 1993 when grunge music was popular and the first web framework was invented — called Common Gateway Interface (CGI) — I was stuck in a horrific dichotomy of simultaneously studying both computer science and celestial navigation at the Naval Academy. I became enamored with the potential of CGI and some of the “newer” ASP web frameworks.

Over the next two decades, I studied and wrote code in some of the best backend web frameworks ever invented, including J2EE (Spring), Tapestry, PHP, Django, Ruby on Rails, and Google Web Toolkit (GWT) to name a few. Each new framework promised to make development easier and faster than ever before. These systems are often considered monolithic because of how developers view them.

A monolithic system is a set of code that operates in a large chunk. The code is written and the entire application is deployed in a single step. There are some great benefits to developing using the monolithic paradigm. Developers have one location to store all of the data and code that they need. Additionally, they can share common objects easily (think users, or widgets) with other parts of the application, and it is simple to deploy – just commit your code, and the entire server is refreshed.

A microservice is much simpler. You only need to develop a single API call for a single task. For example, let’s say you want to develop a simple legal filing application. Rather than build the entire project as a single piece of code, you could break the code into simple single purpose function calls, aka a microservice. In this example, we would build a legal document filing  service that simply takes a person’s first and last name, telephone number, legal paperwork and date of submission. That code would be deployed separately and work independently of the other company software.

The trend: Every blog article extolls the value of building out microservices over traditional monolithic development. Why do something old when you can do something new?

The mistake: I, being the all knowing CTO, with years of experience across multiple tech ventures, decided we were going to build applications using microservices because it was not only the trendy thing to do, but because it made sense on the surface.

Even though our development team was reluctant, they were willing to give it a shot. We were all excited to try something new. We set out diligently to break up our core application into smaller pieces and assign fiefdoms of responsibility that developers could work on independently and with confidence. I figured we could break up the work, operate in parallel and then build our application as fast as possible.

The lessons

This approach failed for the following reasons:

1. Infrastructure — We didn’t invest enough in the infrastructure needed to deploy the microservices. Deploying multiple microservices requires an infrastructure to test and deploy that didn’t exist years ago. AWS has made some strides here, but it still requires focus, effort and a more complex set of policies to enact.
2. Breaking things — While the best developers love the saying “Move Fast, Break Things,” users however don’t always agree … when it comes to breaking things.  Users want to have a reliable experience with little down time or interruption. Careful planning, QA, and great deployment techniques mitigated down time with our monolithic strategy, but we didn’t plan enough for what would happen to our microservices.
3. Sharing Objects — When you write code, there is a good chance that you will need to use some common data models. We tried sharing libraries for the common data models and it became unwieldy very quickly. We didn’t really think through our data model well enough to map data dependencies. With monolithic services, these objects are shared easily and with great performance attributes.

Lessons learned: Take your time upfront with your early build. If you are a well-funded VC company or even if you are bootstrapping your company on nights and weekends, start out by building with your monolithic stack and then, when you hit massive scale needs, make the proper investment in microservices and never look back. Remember, you can always change later. If you hit a scale problem, which 99.9999% of startups will never hit, then you can and should invest in building microservices.

Recovery: The team incorporated all of the services we built out and after some hand wringing, we went back to a more monolithic development model. Since then, we have had very few problems. We did however, keep some high volume microservices around, so the experiment  wasn’t a total loss.

Although the original decision to choose microservices over monolithic turned out to be a mistake, it ultimately helped us recognize what was right for our company and more effectively move forward as a battle tested team.

Learn more about LawIQ and browse jobs