By Henry Scott-Green —

So you want to offer self-hosted deployment: how should you do it?

So you want to offer self-hosted deployment: how should you do it?

Large buyers of software products often request self-hosted deployment, particularly enterprises in regulated industries. Serving these accounts can unlock huge budgets for vendors, but in my experience it’s not easy to do. This post explores the decision of if you should offer a self-hosted deployment model, the key challenges and technical decisions you’ll encounter, and how you can overcome them.

I experienced this problem as co-founder of Context.ai, an LLM product analytics startup. We sold to large enterprises who often didn’t want to share their sensitive data with us, and these customers would ask to self-host our product to ensure their data remained private. After enough prospects asked, we did the work to offer self-hosted deployment - but it introduced many headaches. We now work with software vendors to help them tackle the challenge of deploying and maintaining their products in self-hosted environments.

What is self-hosting and why do customers want it?  

Self-hosted deployments are defined by a vendor's software product running in a customer-controlled environment. The advantage of this is data security, as the vendor usually cannot access the deployment or the customer’s data. But this lack of access means management of the deployment must be completed by the customer rather than the vendor.  

The alternative to self-hosting is cloud SaaS deployment, in which the vendor's software product runs in the vendor's environment. Usually one deployment will support many customers/tenants, but in some cases the vendor will support single tenant cloud hosting too. This model allows for the vendor to manage the deployment, dramatically reducing the operational overhead for both the customer and the vendor, but it comes at the cost of data privacy.

Is self-hosting here to stay?

Vendors are trying to move away from self-hosted deployment. It takes significant support capacity to deliver a worse product experience, and this support cost scales linearly with the number of deployments. This breaks the efficiency of the B2B SaaS business model that requires very low marginal cost per incremental user. Vendors are pushing their customers towards cloud SaaS deployments as a result, but customers are hesitant.

The reality of the market is that customer requirements drive deployment decisions more than vendor preferences. Vendors will largely do what is needed to get a deal across the line. And several groups of customers have persistent requirements to self host: large enterprises; companies in regulated industries such as health, finance, defence, and government; and EU-based organisations. Many vendors I speak to tell me how they’re aggressively pushing customers onto cloud rather than self-hosted deployment, but that >80% of their revenue still comes from their self-hosted deployments.

My conclusion is that the market is moving towards more cloud SaaS. But self-hosting isn’t going away any time soon, particularly for AI products that require access to large amounts of sensitive data.

Key challenges 

Supporting software in an environment you don’t control

  • The primary challenge when supporting self-hosted deployment is that you, as the vendor, are responsible for the success of a product deployed into an environment you can’t control. This adds significant friction to everything you need to do to ensure the success of the deployment. 
  • This is largely not a solvable problem, as the primary motivator for the customer to adopt self-hosted deployment is that the vendor will not have access to the deployment. This causes many downstream challenges, each of which can be mitigated to varying degrees.

Variation across customers

  • Every customer will have their own varied deployment preferences and tech stack. This introduces an enormous challenge when scaling beyond your first handful of deployments. 
  • Best practice here is to be opinionated and confident in your design decisions and push all your customers to one or two models. It will be tempting to accept custom requirements from each new customer in order to get the deal across the line, but this will introduce significant support headaches in future when you have to support dozens of customers each running on different stacks. Some degree of variation is unavoidable, but wherever possible decisions should be imposed on customers. Compelling arguments to push back on customer requirements include improved support through standardization, faster upgrades, quicker deployments, and scalability.

Packaging your application for the first deployment

  • Significant work is required to rearchitect a cloud-hosted product for the first self-hosted deployment, involving a steep learning curve for teams without extensive DevOps experience. This work must be completed up front, and is the most urgent problem when a large customer requests self-hosting in order to sign a large deal. I could write several blog posts just about this challenge, and I will do in future, but some of the challenges to anticipate include: 
  • Containerizing your application using Docker to make it portable and self-contained
  • Replacing dependencies on managed cloud services with self-hosted alternatives, including for services like authentication, analytics, and storage
  • Preparing a deployment process using Helm charts and infrastructure-as-code solutions like Terraform or Ansible 
  • Enabling secure networking with VPNs like Tailscale, configuring ingress and egress, and managing certificates 

Upgrades

  • Regular upgrades are important to ensure customers don’t experience significant version drift, running versions of your application that have issues that were fixed in more recent upgrades. Unfortunately customers are often hesitant to complete upgrades, due to the manual work required and the risk of issues the upgrade process introduces. This introduces challenges around support, as you may have dozens of customers each running different versions - making reproducing reported issues challenging, and meaning customers may still experience known issues that have been fixed in later versions.
  • Good ways to tackle this include automating the upgrade process with enforced best practices such as pre-upgrade validation checks, backups to enable rollbacks, and blue/green deployments. Good documentation and the use of a staging environment are helpful, and some vendors go as far as to contractually mandate upgrades to ensure consistency.

Debugging issues

  • Debugging issues often requires emailing logs back-and-forth, sometimes with lengthy manual redaction requirements. This is a huge problem, with several vendors telling us it takes ~10x longer for a support engineer to reproduce and resolve an issue with a self-hosted deployment compared to a cloud SaaS issue.
  • To solve this, teams often complete Zoom calls to work through issues live, which is expensive but often the most effective solution. Better solutions here look like DataDog’s Agent Flare, pulling a package of logs for easy sharing.
  • A small group of customers are receptive to breakglass access to their deployments for debugging. This grants the vendor temporary access to the deployment, usually via SSH, to debug issues and make changes. This access requires explicit approval from the customer every time, is time-limited, and actions are logged. But many customers won't accept this. 

Delayed time-to-value and longer sales cycles

  • All of the above challenges add up to a significant amount of time taken managing deployment issues that are hidden from the customers in cloud SaaS deployments. This significantly delays the time-to-value for the customer while simultaneously increasing the total cost of ownership. This makes self-hosting unattractive for time-sensitive deployments, such as proof-of-concepts. 

Technical decisions  

Container orchestration

  • We see applications typically deployed using either Kubernetes or Docker Compose - but when should each be used?
  • Kubernetes is the gold standard for orchestrating large distributed systems, and it is the most complex and most powerful choice. It offers the most features for container deployment, scaling, and management across multiple hosts. It is well suited for enterprises who require high availability, multi-region deployments, and self-healing for error recovery. But it comes with significant complexity in setup and maintenance - and a steep learning curve. 
  • Docker Compose is an easier to use but less powerful alternative, gaining simplicity at the expense of scalability. It is limited to single-host deployments, but is good enough for many small to medium-sized applications and proof-of-concepts.  

Configuration management and infrastructure

  • What tooling should you use to help automate your deployments, assuming you’ve used Kubernetes? 
  • Helm is a great choice for Kubernetes configuration management, enabling easy deployment and upgrades of applications deployed on Kubernetes. Helm allows you to define reusable Helm charts for deployments, to maintain a repository of Helm chart versions, and to easily customize the values file for each customer. We always work with Helm, and our first product enables the creation of Helm charts that follow best practices. The alternative to Helm we see most often is Kutomize, but we prefer Helm for its package management and lifecycle management features.
  • Terraform is a great choice when lower level infrastructure needs to be provisioned in customer environments, including the creation of Kubernetes clusters 

What environments will you support?

  • Customers can request deployment into a variety of environments. These start with a virtual private cloud account (e.g. their AWS or GCP), to on-premise (e.g. a data center they control), or fully airgapped environment in which no connection is allowed between the external internet and the deployment. 
  • These introduce significant complexity as you progress through the options, and particularly airgapped deployments introduce a host of challenges around deployment, upgrade, and debugging, and make observability and license management near impossible. 

Vendor or customer managed?

  • Some self-hosted deployments are completed on a vendor-managed basis. In this model, the application is deployed into a customer's environment, but the vendor has access to this environment to manage the environment and complete upgrades and deployments, and to debug issues. This is often referred to as a Bring Your Own Cloud (BYOC) deployment. 
  • This vendor-managed model reduces the support overhead significantly for both the vendor and the customer. But it removes the biggest motivator for self-hosted deployment, which is that the vendor cannot access the environment or the data it contains. Some customers might be happy with this tradeoff, but many are not. 

License management

  • It is challenging to track license validity for self-hosted deployments, as the customer is managing your application outside of your control. This means little stops them technically from continuing to use it without a valid license. Serious companies in Western countries are unlikely to breach license terms like this, but it’s still a risk. 
  • This can be mitigated with license pings back to a vendor-controlled server, reporting usage of the deployment and returning confirmation of the validity of the license.  

User authentication

  • Authentication in cloud SaaS applications is typically completed by managed service auth providers, like all cloud managed services, these are usually not available in self-hosted environments. Developers must then use an alternative solution, typically rolling their own local user account system, with admin roles who can create new accounts and reset passwords. Alternatively, services like Keycloak or Auth0 can be self hosted.

Conclusion: is self-hosting worth it? 

Self-hosted deployment unlocks the highest value deals and some of the best logo customers. But it introduces challenges and significant operational pain. There is meaningful work required to complete your first self-hosted deployment, then high operational cost for ongoing management. All of this is expensive, and takes resources that could be invested in making your core product better. As a result, offering self-hosted deployment is not for the faint of heart. It requires significant and ongoing investment. But that work is justified with high ROI when it enables you to access the biggest deals.

If you’re thinking about if or how you should offer self-hosted deployment please reach out. I’d love to help you work through the challenges, and share what we’ve learned enabling self-hosted deployments for other vendors.