alexsusanu@docs:CKA Guide: ConfigMaps and Secrets - Application Configuration $
alexsusanu@docs
:~$ cat CKA Guide: ConfigMaps and Secrets - Application Configuration.md

HomeNOTES → CKA Guide: ConfigMaps and Secrets - Application Configuration

CKA Guide: ConfigMaps and Secrets - Application Configuration

category: Kubernetes Certification
tags: cka, kubernetes, exam, kubectl, certification

Fundamental Conceptual Understanding

The Configuration Management Philosophy

The Twelve-Factor App Principle:
ConfigMaps and Secrets embody the core principle: "Store config in the environment, not in code"

Traditional Approach (Anti-pattern):
Application Code + Configuration = Monolithic Artifact
├── Hard to change config without rebuilding
├── Different configs require different images  
├── Secrets accidentally committed to version control
└── No separation between dev/staging/prod configs

Kubernetes Approach (Best Practice):
Application Code + External Configuration = Flexible System
├── Same image runs in all environments
├── Configuration changes without rebuilds
├── Secrets managed separately with encryption
└── Environment-specific configs without code changes

The Immutable Infrastructure + Mutable Configuration Pattern:

[Immutable Container Image] + [Mutable ConfigMap/Secret] = [Running Pod]
      (Application Logic)     (Environment-Specific)     (Runtime Instance)

This separation enables:
- Portability: Same image across environments
- Security: Secrets isolated from application code
- Agility: Configuration changes without deployments
- Compliance: Audit trails for sensitive data access

Information Theory Applied: Signal vs Noise

Configuration Data Classification:

Public Configuration (ConfigMaps):
├── Feature flags
├── API endpoints  
├── Timeout values
├── UI themes
└── Non-sensitive environment variables

Private Configuration (Secrets):
├── Database passwords
├── API keys
├── TLS certificates
├── OAuth tokens
└── Encryption keys

The Principle of Least Privilege in Configuration:
Only expose the minimum configuration needed for each component:

Microservice A needs: [DB_HOST, DB_PORT, API_TIMEOUT]
Microservice B needs: [CACHE_URL, LOG_LEVEL, WORKER_COUNT]

Anti-pattern: Give both services access to ALL configuration
Best practice: Each service gets only its required configuration subset

Distributed Systems Security Model

Security Boundaries in Kubernetes:

Cluster Level: RBAC controls who can create/read ConfigMaps/Secrets
Namespace Level: Network policies and resource quotas provide isolation  
Pod Level: ServiceAccounts determine what secrets can be mounted
Container Level: SecurityContext controls file permissions and user context

The Defense in Depth Strategy:
1. Encryption at Rest: etcd encryption for Secrets
2. Encryption in Transit: TLS for API communication
3. Access Control: RBAC for resource permissions
4. Audit Logging: Track who accesses what when
5. Runtime Security: ReadOnly filesystems, non-root users

Core Concepts: ConfigMaps vs Secrets

Architectural Design Patterns

Pattern 1: The Adapter Pattern
ConfigMaps/Secrets act as adapters between your application and the environment:

[Application] ──expects──→ [Standard Interface] ──adapts to──→ [Environment]
     │                           │                              │
  Expects env                ConfigMap/Secret              Kubernetes
  var DATABASE_URL           translates to                environment

Pattern 2: The Strategy Pattern
Different environments use different configuration strategies:

Development Strategy:    ConfigMap with debug settings
Staging Strategy:       ConfigMap with staging endpoints + staging secrets  
Production Strategy:    ConfigMap with prod settings + prod secrets + encryption

Pattern 3: The Template Method Pattern
Base configuration with environment-specific overrides:

base-config.yaml (ConfigMap):
├── shared settings across all environments
├── default timeouts and limits
└── common feature flags

environment-config.yaml (ConfigMap):
├── dev-specific overrides
├── staging-specific overrides  
└── prod-specific overrides

Data Lifecycle Management

ConfigMap Lifecycle Model:

[CREATED] → [MOUNTED] → [UPDATED] → [PROPAGATED] → [CONSUMED]
    ↓           ↓          ↓            ↓            ↓
  kubectl   Pod starts  Config      File/env      App reads
  create    mounts CM   changes     updated       new values

Secret Lifecycle with Security Considerations:

[CREATED] → [ENCRYPTED] → [MOUNTED] → [ACCESSED] → [ROTATED] → [REVOKED]
    ↓           ↓            ↓           ↓           ↓           ↓
  kubectl   Stored in     Pod mounts   App uses    New secret  Old secret
  create    etcd with     as volume    credentials created     deleted
           encryption

ConfigMaps Deep Dive

Creation Patterns and Use Cases

Method 1: Literal Values (Simple Config)

# Single value
kubectl create configmap app-config --from-literal=database_url=postgres://localhost:5432/mydb

# Multiple values  
kubectl create configmap app-config \
  --from-literal=database_url=postgres://localhost:5432/mydb \
  --from-literal=redis_url=redis://localhost:6379 \
  --from-literal=log_level=info

Method 2: From Files (Complex Config)

# From single file
kubectl create configmap nginx-config --from-file=nginx.conf

# From directory (all files become keys)
kubectl create configmap app-configs --from-file=./config-dir/

# From specific file with custom key
kubectl create configmap app-config --from-file=database.properties=./db.conf

Method 3: Declarative YAML (Production Pattern)

apiVersion: v1
kind: ConfigMap
metadata:
  name: application-config
  namespace: production
  labels:
    app: myapp
    environment: production
data:
  # Simple key-value pairs
  database_url: "postgres://prod-db:5432/myapp"
  redis_url: "redis://prod-cache:6379"
  log_level: "warn"

  # Complex configuration files
  application.properties: |
    server.port=8080
    spring.datasource.url=${DATABASE_URL}
    spring.redis.url=${REDIS_URL}
    logging.level.root=${LOG_LEVEL}

  nginx.conf: |
    server {
        listen 80;
        server_name myapp.com;
        location / {
            proxy_pass http://backend:8080;
            proxy_set_header Host $host;
        }
    }

Consumption Patterns in Pods

Pattern 1: Environment Variables (Simple Values)

apiVersion: v1
kind: Pod
metadata:
  name: app-pod
spec:
  containers:
  - name: myapp
    image: myapp:1.0
    env:
    # Single value from ConfigMap
    - name: DATABASE_URL
      valueFrom:
        configMapKeyRef:
          name: app-config
          key: database_url

    # All values from ConfigMap (with optional prefix)
    envFrom:
    - configMapRef:
        name: app-config
      prefix: "APP_"  # Creates APP_database_url, APP_redis_url, etc.

Pattern 2: Volume Mounts (Configuration Files)

apiVersion: v1
kind: Pod  
metadata:
  name: nginx-pod
spec:
  containers:
  - name: nginx
    image: nginx:1.21
    volumeMounts:
    - name: config-volume
      mountPath: /etc/nginx/conf.d
      readOnly: true
    - name: app-config
      mountPath: /app/config
      readOnly: true
  volumes:
  - name: config-volume
    configMap:
      name: nginx-config
      items:
      - key: nginx.conf
        path: default.conf  # File will be created as /etc/nginx/conf.d/default.conf
  - name: app-config
    configMap:
      name: application-config
      defaultMode: 0644  # File permissions

Pattern 3: Hybrid Approach (Best Practice)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: webapp
spec:
  template:
    spec:
      containers:
      - name: app
        image: webapp:1.0
        # Environment variables for simple config
        envFrom:
        - configMapRef:
            name: app-env-config
        # Volume mounts for complex config files
        volumeMounts:
        - name: app-properties
          mountPath: /app/config/application.properties
          subPath: application.properties
          readOnly: true
      volumes:
      - name: app-properties
        configMap:
          name: app-file-config

Secrets Deep Dive

Security-First Design Principles

Principle 1: Minimal Exposure

# Anti-pattern: Mounting entire secret
volumeMounts:
- name: all-secrets
  mountPath: /secrets

# Best practice: Mount only needed keys
volumeMounts:
- name: db-password
  mountPath: /secrets/db-password
  subPath: password
  readOnly: true

Principle 2: Least Privilege Access

# RBAC for secret access
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: secret-reader
rules:
- apiGroups: [""]
  resources: ["secrets"]
  resourceNames: ["myapp-secrets"]  # Only specific secrets
  verbs: ["get", "list"]            # Only read operations

Secret Types and Use Cases

Type 1: Opaque Secrets (Generic)

# From literal
kubectl create secret generic db-secret \
  --from-literal=username=admin \
  --from-literal=password=super-secret-password

# From files
kubectl create secret generic tls-secret \
  --from-file=tls.crt \
  --from-file=tls.key

Type 2: Docker Registry Secrets

# For private container registries
kubectl create secret docker-registry my-registry-secret \
  --docker-server=my-registry.com \
  --docker-username=myuser \
  --docker-password=mypassword \
  --docker-email=myemail@example.com

Type 3: TLS Secrets

# For HTTPS endpoints
kubectl create secret tls tls-secret \
  --cert=path/to/tls.crt \
  --key=path/to/tls.key

Type 4: Service Account Tokens (Automatic)

# Automatically created for ServiceAccounts
apiVersion: v1
kind: ServiceAccount
metadata:
  name: myapp-sa
# Kubernetes automatically creates a secret with JWT token

Advanced Secret Patterns

Pattern 1: Multi-Container Secret Sharing

apiVersion: v1
kind: Pod
metadata:
  name: multi-container-pod
spec:
  containers:
  - name: app
    image: myapp:1.0
    env:
    - name: DB_PASSWORD
      valueFrom:
        secretKeyRef:
          name: db-secret
          key: password
  - name: sidecar
    image: sidecar:1.0
    volumeMounts:
    - name: shared-secret
      mountPath: /shared-secrets
      readOnly: true
  volumes:
  - name: shared-secret
    secret:
      secretName: shared-credentials

Pattern 2: Secret Rotation Strategy

# Blue-Green secret rotation
apiVersion: v1
kind: Secret
metadata:
  name: db-secret-v2  # New version
  labels:
    version: "v2"
    active: "true"
type: Opaque
data:
  password: <new-base64-encoded-password>

---
# Update deployment to use new secret
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
spec:
  template:
    spec:
      containers:
      - name: app
        env:
        - name: DB_PASSWORD
          valueFrom:
            secretKeyRef:
              name: db-secret-v2  # Switch to new version
              key: password

Configuration Management Best Practices

Environment-Specific Configuration Architecture

The Configuration Hierarchy Pattern:

base-config (ConfigMap)           # Shared across all environments
├── default timeouts
├── feature flags (base values)
└── common endpoints

environment-overlay (ConfigMap)   # Environment-specific overrides  
├── dev-config                   
│   ├── debug: true
│   └── log_level: debug
├── staging-config
│   ├── debug: false  
│   └── log_level: info
└── prod-config
    ├── debug: false
    └── log_level: warn

secrets-per-environment (Secret)  # Isolated secrets
├── dev-secrets
├── staging-secrets  
└── prod-secrets

Implementation with Kustomize:

# base/kustomization.yaml
resources:
- configmap.yaml
- deployment.yaml

# overlays/production/kustomization.yaml  
bases:
- ../../base
configMapGenerator:
- name: app-config
  literals:
  - log_level=warn
  - debug=false
secretGenerator:
- name: app-secrets
  files:
  - db-password=prod-db-password.txt

Immutable ConfigMaps Pattern

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config-v1  # Versioned name
immutable: true        # Cannot be changed
data:
  version: "1.0"
  config: "initial config"

---
# To update: create new ConfigMap, update deployment
apiVersion: v1  
kind: ConfigMap
metadata:
  name: app-config-v2  # New version
immutable: true
data:
  version: "2.0" 
  config: "updated config"

Benefits of Immutable ConfigMaps:
- Prevents accidental changes in production
- Enables atomic updates via deployment changes
- Improves etcd performance (no watch events)
- Simplifies rollback (just reference old ConfigMap)

Configuration Update Strategies

Hot Reload vs Restart Strategies

Strategy 1: Application Hot Reload

# Applications that detect file changes
apiVersion: v1
kind: Pod
spec:
  containers:
  - name: app
    image: nginx:1.21  # nginx auto-reloads on config change
    volumeMounts:
    - name: config
      mountPath: /etc/nginx/conf.d
  volumes:
  - name: config
    configMap:
      name: nginx-config

Strategy 2: Restart-Required Applications

# Update ConfigMap
kubectl patch configmap app-config -p '{"data":{"log_level":"debug"}}'

# Force pod restart to pick up changes
kubectl rollout restart deployment/myapp

Strategy 3: Blue-Green Configuration Updates

# Create new ConfigMap version
kubectl create configmap app-config-v2 --from-literal=log_level=debug

# Update deployment to use new ConfigMap
kubectl patch deployment myapp -p '{
  "spec": {
    "template": {
      "spec": {
        "volumes": [{
          "name": "config",
          "configMap": {"name": "app-config-v2"}
        }]
      }
    }
  }
}'

Security Deep Dive

Secret Encryption Architecture

Encryption at Rest Model:

kubectl create secret → API Server → etcd (encrypted) → Persistent Storage
                           ↓
                      Encryption Key
                           ↓
                    Key Management Service
                      (AWS KMS, etc.)

Encryption in Transit:

kubectl → HTTPS/TLS → API Server → gRPC/TLS → etcd
                                       ↓
                                 Internal TLS → kubelet → Pod

RBAC for Configuration Security

# Least privilege for ConfigMap access
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: config-reader
rules:
- apiGroups: [""]
  resources: ["configmaps"]
  resourceNames: ["app-config", "nginx-config"]
  verbs: ["get", "list"]

---
# More restrictive for Secrets
apiVersion: rbac.authorization.k8s.io/v1
kind: Role  
metadata:
  name: secret-reader
rules:
- apiGroups: [""]
  resources: ["secrets"]
  resourceNames: ["app-secrets"]  # Specific secrets only
  verbs: ["get"]                  # Read-only

Secret Scanning and Compliance

GitOps Security Pattern:

# Pre-commit hooks to prevent secret commits
# .pre-commit-config.yaml
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
  hooks:
  - id: detect-private-key
  - id: detect-aws-credentials

# Secret scanning in CI/CD
- name: Run secret scan
  run: |
    truffleHog --regex --entropy=False .
    detect-secrets scan --all-files

Troubleshooting Configuration Issues

Systematic Debugging Framework

Layer 1: Configuration Source Validation

# Verify ConfigMap/Secret exists and has correct data
kubectl get configmap app-config -o yaml
kubectl get secret app-secret -o yaml

# Check for encoding issues in Secrets
kubectl get secret app-secret -o jsonpath='{.data.password}' | base64 -d

Layer 2: Mount Point Analysis

# Check if volume is mounted correctly
kubectl describe pod myapp-pod | grep -A 10 "Mounts:"

# Exec into pod to verify files
kubectl exec -it myapp-pod -- ls -la /app/config/
kubectl exec -it myapp-pod -- cat /app/config/application.properties

Layer 3: Environment Variable Verification

# Check environment variables in running pod
kubectl exec -it myapp-pod -- env | grep DATABASE

# Verify environment variable source
kubectl describe pod myapp-pod | grep -A 5 "Environment:"

Layer 4: Application Configuration Debugging

# Check application logs for config parsing errors
kubectl logs myapp-pod | grep -i "config\|error\|fail"

# Verify application is reading the correct config
kubectl exec -it myapp-pod -- curl localhost:8080/actuator/configprops

Common Anti-Patterns and Solutions

Anti-Pattern 1: Hardcoded Configuration

# Wrong: Configuration in container image
containers:
- name: app
  image: myapp:1.0  # Contains hardcoded database URL

# Right: External configuration
containers:
- name: app  
  image: myapp:1.0
  env:
  - name: DATABASE_URL
    valueFrom:
      configMapKeyRef:
        name: app-config
        key: database_url

Anti-Pattern 2: Secrets in ConfigMaps

# Wrong: Sensitive data in ConfigMap
apiVersion: v1
kind: ConfigMap
data:
  database_password: "super-secret"  # Visible in plain text!

# Right: Sensitive data in Secret
apiVersion: v1
kind: Secret
type: Opaque
data:
  database_password: c3VwZXItc2VjcmV0  # Base64 encoded

Anti-Pattern 3: Overly Broad Secret Access

# Wrong: Mounting entire secret directory
volumeMounts:
- name: all-secrets
  mountPath: /secrets  # Exposes ALL secrets

# Right: Specific secret mounting
volumeMounts:
- name: db-password
  mountPath: /secrets/db-password
  subPath: password  # Only the needed key

Exam Tips & Quick Reference

⚡ Essential Commands for Exam

# Quick ConfigMap creation
kubectl create cm app-config --from-literal=key=value --dry-run=client -o yaml

# Quick Secret creation  
kubectl create secret generic app-secret --from-literal=password=secret123

# View decoded secret
kubectl get secret app-secret -o jsonpath='{.data.password}' | base64 -d

# Update ConfigMap
kubectl patch configmap app-config -p '{"data":{"key":"new-value"}}'

# Create from file with custom key name
kubectl create configmap nginx-config --from-file=nginx.conf=./my-nginx.conf

# Mount specific keys only
# Use subPath in volumeMounts to mount individual files

🎯 Exam Scenarios and Solutions

Scenario 1: Application Configuration

# Create ConfigMap for application settings
kubectl create configmap webapp-config \
  --from-literal=database_host=mysql-service \
  --from-literal=redis_host=redis-service \
  --from-literal=log_level=info

# Create Secret for sensitive data
kubectl create secret generic webapp-secret \
  --from-literal=db_password=mysecretpassword \
  --from-literal=api_key=abc123xyz

Scenario 2: File-based Configuration

# Create ConfigMap from configuration file
kubectl create configmap nginx-config --from-file=nginx.conf

# Create Secret from certificate files
kubectl create secret tls tls-secret --cert=tls.crt --key=tls.key

🚨 Critical Gotchas

  1. Base64 Encoding: Secrets are base64 encoded, not encrypted
  2. File Permissions: Default is 0644, may need to set explicitly
  3. Immutable Flag: Once set to true, ConfigMap cannot be changed
  4. SubPath Mounts: Don't get automatic updates when ConfigMap changes
  5. Environment Variable Updates: Require pod restart to take effect
  6. Secret Size Limit: Maximum 1MB per secret
  7. etcd Storage: All data counts against etcd size limits

WHY This Matters - The Deeper Philosophy

Software Engineering Principles Applied

1. The Single Responsibility Principle:

Application Code: "How to process requests"
Configuration: "Where to connect and what settings to use"  
Secrets: "Credentials needed for secure access"

Each component has a clear, separate responsibility.

2. The Open/Closed Principle:
Applications are:
- Open for configuration: Behavior changes via external config
- Closed for modification: Code doesn't change between environments

3. Dependency Inversion Principle:

High-level Application ──depends on──→ Configuration Interface
                                            ↑
                                     ConfigMap/Secret
                                    (Implementation Detail)

Information Security Theory

The CIA Triad Applied:
- Confidentiality: Secrets protect sensitive information
- Integrity: Immutable ConfigMaps prevent unauthorized changes
- Availability: External configuration enables environment portability

Defense in Depth Strategy:

Layer 1: RBAC (Who can access)
Layer 2: Namespace isolation (What can be accessed)  
Layer 3: Encryption at rest (How data is stored)
Layer 4: Encryption in transit (How data moves)
Layer 5: Application-level validation (How data is used)

Systems Architecture Philosophy

The Configuration Pyramid:

                    [Secrets]              ← Highest Security
                 [Environment Config]     ← Environment-specific  
              [Feature Flags & Settings]  ← Application behavior
           [Infrastructure Configuration] ← Deployment settings
        [Base Application Configuration]  ← Fundamental settings

Each layer builds on the previous, with increasing security requirements as you go up.

Event-Driven Configuration:

Config Change → ConfigMap Update → Pod Mount Update → 
Application Reload → Service Behavior Change → 
Monitoring Alert → Operator Notification

Production Engineering Insights

The Configuration Management Maturity Model:

Level 1: Basic
- Hard-coded configuration in images
- Manual secret management
- No environment separation

Level 2: Separated
- ConfigMaps for non-sensitive data
- Basic secrets for credentials
- Some environment separation

Level 3: Systematic
- Immutable configuration patterns
- Automated secret rotation
- Environment-specific configurations
- GitOps for configuration management

Level 4: Advanced
- Dynamic configuration with hot reload
- Secret scanning and compliance
- Configuration validation and testing
- Automated configuration drift detection

Career Development Implications

For DevOps Engineers:
- Configuration as Code: Treat configuration with same rigor as application code
- Security Mindset: Always consider the security implications of configuration choices
- Operational Excellence: Design for observability and troubleshooting

For Software Architects:
- Separation of Concerns: Design applications that externalize configuration properly
- Portability: Enable applications to run anywhere with proper configuration
- Security by Design: Build security considerations into configuration architecture

For the Exam:
- Practical Skills: You'll create, mount, and troubleshoot configuration
- Security Understanding: Demonstrate knowledge of secrets vs configmaps
- Troubleshooting: Debug configuration-related application failures
- Best Practices: Show understanding of proper configuration management

For Production Systems:
- Reliability: Proper configuration management prevents outages
- Security: Correct secret handling prevents data breaches
- Agility: External configuration enables rapid environment changes
- Compliance: Proper secret management meets regulatory requirements

Understanding ConfigMaps and Secrets deeply teaches you how to build secure, portable, and maintainable applications - skills that are essential for any modern infrastructure role and crucial for passing the CKA exam.

Last updated: 2025-08-26 20:00 UTC