Observations and thoughts after building 3 kubernetes platforms in Financial Services Industry — Part 4 — Policy Enforcement
In the fourth part of this blog series, I share my observations in the policy enforcement space.
Previous parts:
- Part 1 — Overview, K8s PaaS offerings, Networking
- Part 2 — Workload Identity, Secret Management / Externalization
- Part 3 — GitOps
Policy Enforcement
Policy enforcement and controls management is a key area in building a secure Kubernetes platform in regulated environments. There is a panoply of open source tools (OPA Gatekeeper, Kyverno, Trivy, Falco) and paid solutions (Aquasec, Twistlock). This solution is far from complete; feel free to look at the CNCF solution map.
My first observation is that organization struggle picking a tool; they want all of them. I recommend starting with a needs matrix and align the tools with the needs. Pick the best match.
AVOID overlapping multiple tools over the same control space.
You will end up with duplicated controls (effort) or discrepancies in implementations leading to user frustration (system X denied me error X, system Y denied me with error Y, and so on; resulting in death by a thousand paper cuts).
Regulated environments love buying tools; especially in the security space. Aquasec and Twistlock are very mature tools and cover a wide swath of controls. I consider this space as saturated & mature; therefore you would be well served by either. However, organizations purchase the tool, but don’t spend enough effort to institutionalizing and operationalizing them. The tools are as useful as the effort you put into managing them. Cloud providers are slowly eating into this space by adding more and more controls to the base platform. Prime example is Microsoft Defender for Containers.
Open Source tools such as OPA Gatekeeper are mature; but require specialized development skillset. Developing policies in Rego language is not for everyone. Cloud providers often provide a policy library that you can bootstrap your platform controls. However, sooner than later, you will need to create your own policies. This means establishing a development lifecycle, policy testing, and release management. If your security team does not have software development skills, this will be a significant hurdle to adoption.
Lastly, Cloud Providers will usually have value adds in this space:
- Azure Policy providing a single pane to view OPA Gatekeeper auditing results & centralized management
- OPA Gatekeeper as a managed service in either platform (Anthos Policy Controller on GKE, Azure Policy on AKS). Both provide a base set of policies to use.
Key Observations:
- This is an area where buying over build may be the better option
- Buying a tool is not enough; like any technology it requires care and feeding
- You will need multiple tools to cover the full spectrum; avoid overlaps. Where there is overlap, pick a tool for the problem and avoid duplication.
- Expose the results to your tenants. Central teams don’t have time to chase everything. Shift left and put the accountability where things can be addressed.
- Defense in depth is important
- If you have the skills, look at OPA & Kyverno. Make sure to look at existing libraries before developing new policies.
