In the part-3 of the build your own serverless series, we went over adding sqlite using Gorm, using a new hostname strategy, docker go client instead of exec.Command, and we added graceful shutdown to clean up resources.
In this post we will be covering advanced topics:
Support of versioning
Version Promotion (Canary, A/B & more)
Garbage collection of idle containers
Follow along to learn these advanced topics and take our serverless platform to the next level!
Subscribe for free to unlock exclusive content!
Versioning & Environment Variables
Versioning is a powerful capability that will enable us to easily create and track new versions of our Cless containers. Each version will have a clear relationship to our overall service definition, giving us full visibility into what container versions are currently running as well as a history of how our service has evolved over time.
Previously we put the serverless resource into our ServiceDefinition object, which contained all the necessary attributes to start a container and keep track of it.
Instead we will extract the service container attributes into a new struct named ServiceVersion and add property EnvVars as datatypes.JSONSlice[string] , which will store a container environment variables like ENV=prod:
Now we need to establish a relationship of has_many between ServiceDefinition and ServiceVersion:
Methods & Persistence:
Now that types has been clearly defined we will need to refactor the isValid method on ServiceDefinition and add same method to ServiceVersion:
Let's extend our repository definition to support persistence of new versions:
In our SqliteServiceDefinitionRepository we will implement the new function AddVersion, we use Gorm Associations to make managing relationship easier:
We need a rest api for managing ServiceVersion similar to how we did before for ServiceDefinition, we need a function on ServiceDefinitionManager cause we don't want to be calling the repository directly from our http server route:
within StartAdminServer we will add ServiceVersion routes:
As you can see with minimal refactoring we were able to incorporate versions in cLess, but that is only like one third of the story we still need a way to promote new versions to be served, and that's why we will need a concept for the proxy to know which version to serve.
Version Promotion (Canary, A/B & more):
We will implement powerful versioning capabilities that go beyond just pinning a versionID to ServiceDefinition. We can actually split traffic between multiple versions with precise weighting - enabling advanced deployment patterns like canary testing. Rather than being limited to routing 100% of traffic to a single version, we can divide traffic across versions as needed - sending 1% to a new version for incremental testing, 50/50 split to compare A/B, or any ratio we choose. This unlocks fine-grained control over our version rollout strategy.
Let's create a struct TrafficWeight that will contains traffic split definitions and reference it in ServiceDefinition:
we will use a has_many relationship between ServiceDefinition and TrafficWeight. This provides full historical traceability of how traffic has been split across versions over time. If a particular traffic split causes issues, we can quickly revert back to a known good distribution. The history of TrafficWeights gives us an audit trail showing how traffic shifted as new versions rolled out. And it allows us to easily recreate previous traffic splitting strategies. This time-travel capability ensures risk-free rollbacks while freeing us to confidently experiment with different version promotion patterns. The combination of fine-grained traffic splitting and historical TrafficWeights gives us unparalleled control over our version rollout timeline.
Method for validating TrafficWeight:
Let's add persistence as well similar to what we did with ServiceVersion:
To manage these traffic weights we will need a rest api and an AddTrafficWeight in our ServiceDefinitionManager:
Now that we have the ability to store and manage traffic weights, let's add the ability to use them. We will implement a method called ChooseVersion, which will help pick a version from the latestTrafficWeight of a ServiceDefinition.
We will see later how to use this method in main.go.
With all these changes, minimal refactoring will also be needed in the DockerContainerManager. Currently it relies on ServiceDefinition, so instead we will create a struct called ExternalServiceDefinition, which will help integrate with other modules and keep the admin module loosely coupled with the container module.
Now we just need to replace ServiceDefinition with ExternalServiceDefinition in DockerContainerManager, the change isn't hard, here is an example for GetRunningServiceForHost:
The only thing that will change in our main.go is the http handler:
We have already added the EnvVars property in ServiceVersion above, here we will cover how to add it to the container that is going to be started by the container manger:
Not a lot was needed as ServiceVersion has the same format for env variable as what's required by the go docker client api. And we just do a pass through to the container.Config.
Garbage Collection of Idle Containers
Since cLess is a serverless platform we should have a way to stop containers that are idle -which haven't received traffic for a specific amount of time; for instance, we can say that if containers haven't received traffic for 2 minutes we can garbage collect them.
for that we need a way to track the last time a RuningService has been accessed:
the LastTimeAccessed property will keep track of last time the running service served traffic.
We will also need to keep it updated, we will do that in GetRunningServiceForHost, since that method gets called every time the service gets accessed:
and running docker ps confirm that no idle container is running.
With versioning, we were able to iteratively improve our cLess platform while maintaining governance over our container portfolio.Upgrading becomes simplified as we can promote validated versions into production with minimal effort. Versioning unlocks new levels of agility, control and audit for our critical cLess applications.
we can now validate new versions with a portion of live traffic before ramping up. Or roll back issues by shifting traffic away from faulty versions. This will give us the advanced tools to implement the versioning workflows that best suit our use case. The end result is lower risk deployments and greater application stability.
Now cLess provides an automatic garbage collector for idle containers, unlocking new optimization capabilities. When container instances are inactive for a specified time period, cLess will gracefully scale them to zero. This allows us to run lean - paying only for the compute we need at any given moment. As demand ramps up, cLess will seamlessly spin a new container to server traffic. The end result is maximized efficiency and minimized waste.