Crossplane Fundamentals⚓︎

Introduction to Crossplane⚓︎

What is Crossplane?⚓︎

Crossplane is an open-source Cloud Native Computing Foundation (CNCF) project that transforms Kubernetes into a universal control plane for managing infrastructure and services across multiple cloud providers and platforms.

At its core, Crossplane extends Kubernetes by:

Adding custom resource definitions (CRDs) for infrastructure
Providing controllers that reconcile infrastructure state
Enabling declarative infrastructure management using YAML
Creating a consistent API across different cloud providers

Key Capabilities⚓︎

Crossplane enables you to:

Manage Cloud Resources: Provision and manage resources from AWS, GCP, Azure, and more using Kubernetes-native tools
Define Infrastructure as Code: Use YAML manifests to declare infrastructure, just like you do for applications
Build Platform Abstractions: Create custom APIs that hide complexity and provide self-service capabilities
Unify Multi-Cloud Management: Manage resources across multiple clouds using a single control plane
Enable GitOps: Version control your infrastructure and use GitOps workflows for deployment

Why Crossplane?⚓︎

The Problem with Traditional Infrastructure Management Traditional infrastructure management faces several challenges:

Multiple Tools: Each cloud provider has its own CLI, SDK, and management tools (AWS CLI, gcloud, az cli)
Fragmented Workflows: Application deployment uses Kubernetes, but infrastructure uses separate tools like Terraform, CloudFormation, or ARM templates
Limited Self-Service: Developers often need to request infrastructure from operations teams, creating bottlenecks
Inconsistent APIs: Each cloud provider has different resource schemas and APIs, making multi-cloud difficult
Complex Credential Management: Managing cloud credentials securely across teams is challenging

How Crossplane Solves These Problems⚓︎

Unified Control Plane: Kubernetes becomes the single control plane for both applications and infrastructure
Consistent API: All infrastructure resources use Kubernetes-native APIs, regardless of cloud provider
Declarative Management: Infrastructure is declared in YAML and reconciled continuously, just like Kubernetes workloads
GitOps-Ready: Infrastructure manifests can be stored in Git and deployed using GitOps tools like ArgoCD or Flux
Self-Service Platform: Platform teams can create abstractions that enable developers to provision infrastructure safely
Native RBAC: Use Kubernetes Role-Based Access Control to manage who can create what infrastructure

Core Architecture⚓︎

Component Breakdown

Crossplane Core: The main controller that manages the Crossplane lifecycle and core CRDs
RBAC Manager: Handles role-based access control for Crossplane resources
Providers: Pluggable controllers that manage resources for specific platforms (AWS, GCP, Azure, Kubernetes, etc.)
CRDs: Define the schemas for all Crossplane resources

Key Components⚓︎

Control Plane

The Crossplane Control Plane is the brain of the system. It consists of:

Core Controller:
- Watches Crossplane custom resources in the Kubernetes API
- Manages the lifecycle of Providers, Compositions, and XRDs
- Ensures the system is operating correctly
Reconciliation Loop:
- Continuously compares desired state (what you declared) vs actual state (what exists)
- Takes actions to make actual state match desired state
- Handles create, update, and delete operations
Key Responsibilities:
- Install and manage Providers
- Create and manage CRDs
- Coordinate between different Crossplane components
- Handle errors and retries
Custom Resource Definitions (CRDs)

What are CRDs?

Custom Resource Definitions are Kubernetes' extension mechanism. They allow you to define new resource types beyond the built-in types (Pods, Services, Deployments, etc.).

How Crossplane Uses CRDs:

Crossplane installs CRDs for:
- Core Types: Provider, ProviderConfig, CompositeResourceDefinition, Composition, Function
- Provider-Specific Types: Each provider adds CRDs for its resources (e.g., Bucket, Instance, Database)
Example CRD Usage:

Comment # After installing AWS Provider, you get CRDs like:
1 2 3 4 5 6 7 8
apiVersion: s3.aws.crossplane.io/v1beta1 kind: Bucket metadata: name: my-bucket spec: forProvider: region: us-east-1 acl: private
Why CRDs Matter:
- Make infrastructure manageable with kubectl
- Enable declarative infrastructure definitions
- Integrate with Kubernetes RBAC, admission controllers, and operators
- Allow infrastructure to be treated like any other Kubernetes resource
Providers

What is a Provider?

A Provider is a Crossplane package that extends Crossplane with support for managing resources on a specific platform.

Provider Structure:

Each provider contains:
- CRDs: Define the resource types available (e.g., Bucket, Instance)
- Controller: A pod that watches those resources and reconciles them
- Metadata: Information about the provider (version, dependencies)
Common Providers:
- provider-aws: Manage AWS resources (S3, EC2, RDS, VPC, etc.)
- provider-gcp: Manage Google Cloud resources (GCS, GCE, Cloud SQL, etc.)
- provider-azure: Manage Azure resources (Blob Storage, VMs, SQL, etc.)
- provider-kubernetes: Manage Kubernetes resources
- provider-helm: Manage Helm releases
- provider-sql: Manage SQL databases
How Providers Work:
1. You create a managed resource (e.g., S3 Bucket)
2. Provider controller watches for that resource
3. Controller makes API calls to AWS to create the bucket
4. Controller updates the resource status in Kubernetes
5. Controller continuously reconciles to ensure state matches
Provider Installation:

Providers are installed as Kubernetes resources:
1 2 3 4 5 6
apiVersion: pkg.crossplane.io/v1 kind: Provider metadata: name: provider-aws spec: package: xpkg.upbound.io/crossplane-contrib/provider-aws:v0.40.0
When applied, Crossplane:
- Downloads the provider package (OCI image)
- Extracts and installs the CRDs
- Deploys the provider controller pod
- Makes the provider ready to use

Managed Resources (MRs)

What is a Managed Resource?

A Managed Resource is a Kubernetes custom resource that represents a single piece of infrastructure in an external system.

Characteristics:

Can be either cluster-scoped or namespaced (namespaced support added in v2)
Maps 1:1 with an external resource
Contains full configuration for the resource
Provides status information about the resource

Example - AWS S3 Bucket:

apiVersion: s3.aws.crossplane.io/v1beta1
kind: Bucket
metadata:
  name: my-app-data-bucket
spec:
  forProvider:
    region: us-west-2
    acl: private
    versioningConfiguration:
      status: Enabled
    serverSideEncryptionConfiguration:
      rules:
        - applyServerSideEncryptionByDefault:
            sseAlgorithm: AES256
  providerConfigRef:
    name: aws-provider-config

Managed Resource Lifecycle:

Creation: When you apply the YAML, Crossplane creates the resource in the external system
Reconciliation: Crossplane continuously checks that the actual resource matches the desired state
Updates: Changes to the spec are applied to the external resource
Deletion: When you delete the resource, Crossplane deletes it from the external system

Status Fields:

Managed resources provide status information:

status:
  conditions:
    - type: Ready
      status: "True"
      reason: Available
    - type: Synced
      status: "True"
      reason: ReconcileSuccess
  atProvider:
    bucketArn: arn:aws:s3:::my-app-data-bucket
    bucketDomainName: my-app-data-bucket.s3.amazonaws.com

ProviderConfig

What is a ProviderConfig?

A ProviderConfig defines authentication credentials and configuration for a provider.

Purpose:
- Tells the provider how to authenticate to the external platform
- Can define different configurations for different environments
- Referenced by managed resources to specify which credentials to use
Example - AWS Provider:
```
apiVersion: aws.crossplane.io/v1beta1
kind: ProviderConfig
metadata:
  name: aws-production
spec:
  credentials:
    source: Secret
    secretRef:
      namespace: crossplane-system
      name: aws-credentials
      key: credentials
```
Multiple ProviderConfigs:

You can have multiple ProviderConfigs for different purposes:
- Different AWS accounts (dev, staging, prod)
- Different GCP projects
- Different regions
- Different credential types
Namespaces in Crossplane

crossplane-system Namespace:

Crossplane is installed in the crossplane-system namespace by default.

What's in this namespace?
- Crossplane controller pods
- RBAC manager pods
- Provider controller pods
- Secrets for provider credentials
- ConfigMaps for configuration
Why a Dedicated Namespace?
- Isolation: Separates Crossplane components from other workloads
- RBAC: Easier to manage permissions for Crossplane
- Resource Management: Apply resource quotas and limits
- Organization: Clear separation of concerns
Viewing Resources:
```
# See all pods in crossplane-system
kubectl get pods -n crossplane-system
```
```
# See all resources
kubectl get all -n crossplane-system
```

How Crossplane Works⚓︎

The Reconciliation Loop

Crossplane uses Kubernetes' controller pattern with a reconciliation loop:

Workflow Example: Creating an S3 Bucket Let's walk through what happens when you create an S3 bucket:

User Creates Resource
```
kubectl apply -f s3-bucket.yaml
```
Kubernetes Stores Resource
- Resource is stored in etcd
- Kubernetes API server notifies watchers
Provider Controller Detects Change
- AWS provider controller watches for Bucket resources
- Sees the new bucket resource
Controller Reconciles
- Reads the bucket spec
- Checks if bucket exists in AWS
- Bucket doesn't exist, so create it
Controller Calls AWS API

CreateBucket(
  Bucket: "my-app-data-bucket",
  Region: "us-west-2",
  ACL: "private"
)

Update Resource Status
Update conditions: Ready=True, Synced=True
Add AWS-specific details (ARN, domain name)
Continuous Monitoring
Controller periodically checks bucket still exists
Detects any drift (manual changes in AWS console)
Reconciles back to desired state

Resource Dependencies

Crossplane handles resource dependencies automatically:

Example: VPC with Subnet

# VPC created first
apiVersion: ec2.aws.crossplane.io/v1beta1
kind: VPC
metadata:
  name: my-vpc
spec:
  forProvider:
    cidrBlock: 10.0.0.0/16

---
# Subnet references VPC
apiVersion: ec2.aws.crossplane.io/v1beta1
kind: Subnet
metadata:
  name: my-subnet
spec:
  forProvider:
    cidrBlock: 10.0.1.0/24
    vpcIdRef:
      name: my-vpc  # Reference to VPC resource

Crossplane ensures:

VPC is created first
Subnet waits for VPC to be ready
Subnet uses actual VPC ID from AWS
If VPC is deleted, subnet is deleted first

Installation Methods⚓︎

Helm Installation (Recommended) Helm is the recommended way to install Crossplane.

Steps:

# Add Crossplane Helm repository
helm repo add crossplane-stable https://charts.crossplane.io/stable
helm repo update

# Install Crossplane
helm install crossplane crossplane-stable/crossplane \
  --namespace crossplane-system \
  --create-namespace

Benefits:

Easy upgrades with helm upgrade
Configurable via values.yaml
Standard Kubernetes deployment method
Easy to manage versions

Common Configurations:

values.yaml

replicas: 2  # High availability

resources:
  limits:
    cpu: 500m
    memory: 512Mi

args:
  - --enable-composition-functions  # Enable composition functions (GA in v2)

Kubectl Installation

You can also install using kubectl:

kubectl create namespace crossplane-system
kubectl apply -f https://raw.githubusercontent.com/crossplane/crossplane/master/cluster/charts/crossplane/crds

Verification

After installation, verify:

# Check pods are running
kubectl get pods -n crossplane-system

# Should see:
# crossplane-xxx
# crossplane-rbac-manager-xxx

# Check CRDs are installed
kubectl get crds | grep crossplane

# Check Crossplane version
kubectl get deployment crossplane -n crossplane-system -o yaml | grep image:

Best Practices⚓︎

Namespace Organization
- Use dedicated namespace: Always install Crossplane in crossplane-system for consistency
- Separate credentials: Store provider credentials in Secrets within crossplane-system
- Resource isolation: Keep Crossplane components isolated from application workloads
Version Management
- Pin versions: Always specify explicit provider versions
```
spec:
  package: provider-aws:v0.40.0  # Good
  # package: provider-aws:latest  # Avoid
```
- Test upgrades: Test Crossplane upgrades in non-production first
- Monitor compatibility: Check provider compatibility with Crossplane version
Security
- Least privilege: Use minimal IAM/RBAC permissions required
- Credential rotation: Rotate provider credentials regularly
- Network policies: Apply network policies to crossplane-system namespace
- Audit logging: Enable audit logging for infrastructure changes
Monitoring
- Watch pod health: Monitor Crossplane and provider pod health
```
kubectl get pods -n crossplane-system -w
```
- Check resource status: Regularly check managed resource conditions
```
kubectl get managed
```
- Log aggregation: Collect logs from Crossplane pods for troubleshooting
Resource Management
- Set resource limits: Configure CPU and memory limits for controllers
- Scale appropriately: Increase replicas for high availability
- Clean up unused resources: Delete unused managed resources to avoid costs
Development Workflow
- Use GitOps: Store all Crossplane manifests in Git
- Environment separation: Use different ProviderConfigs for dev/staging/prod
- Testing: Test infrastructure changes in development environments first
- Documentation: Document your custom compositions and resource definitions

Common Commands Reference⚓︎

Installation and Setup⚓︎

# Install Crossplane
helm install crossplane crossplane-stable/crossplane \
  --namespace crossplane-system \
  --create-namespace

# Install Crossplane CLI
curl -sL "https://raw.githubusercontent.com/crossplane/crossplane/master/install.sh" | sh
sudo mv crossplane /usr/local/bin

Checking Status⚓︎

# Check all Crossplane pods
kubectl get pods -n crossplane-system

# Check installed CRDs
kubectl api-resources | grep crossplane

# Check providers
kubectl get providers

# Check all managed resources
kubectl get managed

# Get detailed resource info
kubectl describe <resource-type> <resource-name>

Troubleshooting⚓︎

# View controller logs
kubectl logs -n crossplane-system deployment/crossplane

# View provider logs
kubectl logs -n crossplane-system deployment/<provider-name>

# Check resource events
kubectl describe <resource-type> <resource-name>

# Check resource conditions
kubectl get <resource-type> <resource-name> -o yaml | grep -A 10 conditions

Summary⚓︎

Crossplane transforms Kubernetes into a universal control plane for infrastructure management:

Extends Kubernetes with CRDs for infrastructure resources
Uses Providers to manage resources across different platforms
Employs reconciliation loops to maintain desired state
Enables declarative infrastructure using YAML manifests
Supports GitOps workflows for infrastructure as code
Provides platform abstraction through compositions