Secure AI Agent Best Practices - Part 4: Secure API Key Management

API keys are the lifeblood of modern AI agents - they provide access to language models, external services, databases, and third-party integrations. However, compromised API keys can result in devastating consequences: unauthorized access to AI services, massive unexpected bills, data breaches, and complete system compromise.

This comprehensive guide will teach you to implement enterprise-grade API key management that protects your keys throughout their entire lifecycle, from generation to rotation to secure disposal.

Why API Key Security is Critical for AI Agents

Unique API Key Challenges for AI Agents AI agents face API key security challenges that traditional applications don't encounter:

High-Value Targets AI API keys (OpenAI, Anthropic, etc.) can generate significant costs if misused. A compromised OpenAI API key could result in thousands of dollars in unauthorized usage within hours.

Multiple Service Dependencies AI agents typically use 5-15 different APIs (AI services, databases, messaging platforms, analytics tools). Each API key represents a potential attack vector.

Long-Running Operations AI agents often run continuously, requiring persistent access to API keys. This increases exposure time compared to short-lived applications.

Cross-Environment Usage API keys must work across development, staging, and production environments while maintaining different security levels for each.

What You'll Learn in This Tutorial

By the end of this tutorial, you'll have:

• ✅ Comprehensive key management system with secure storage and access control
• ✅ Automated key rotation strategies that minimize downtime
• ✅ Multi-environment key management with proper isolation
• ✅ Key usage monitoring and anomaly detection
• ✅ Incident response procedures for compromised keys
• ✅ Compliance frameworks for enterprise security requirements

Estimated Time: 40-45 minutes

---

Step 1: Understanding API Key Security Fundamentals

Before implementing key management, it's crucial to understand the security principles that govern API key handling.

API Key Security Principles

Principle of Least Privilege Each API key should have the minimum permissions necessary for its intended function. Don't use admin keys for routine operations.

Defense in Depth Multiple security layers protect API keys:

• Storage encryption (keys encrypted at rest)
• Transport encryption (keys encrypted in transit)
• Access controls (who can access keys)
• Usage monitoring (detecting unusual activity)
• Regular rotation (limiting exposure time)

Separation of Concerns Different environments (dev, staging, production) should use completely separate API keys to prevent cross-environment contamination.

Common API Key Vulnerabilities

Hard-Coded Keys

// ❌ NEVER DO THIS - Keys in source code
const openai = new OpenAI({
    apiKey: 'sk-1234567890abcdef...' // This will be exposed in version control
});

// ✅ CORRECT - Keys from environment variables
const openai = new OpenAI({
    apiKey: process.env.OPENAI_API_KEY
});

Overprivileged Keys Using admin-level API keys for routine operations creates unnecessary risk. If the key is compromised, attackers have excessive access.

Stale Keys Old, unused API keys that aren't properly deactivated create security vulnerabilities without providing any value.

Insufficient Monitoring Without proper monitoring, compromised keys can be used for weeks or months before detection.

---

Step 2: Implementing Secure Key Storage

Let's build a comprehensive key management system that handles storage, encryption, and access control.

Enterprise Key Management System

// security/key-manager.js

const crypto = require('crypto');
const fs = require('fs').promises;
const path = require('path');

class SecureKeyManager {
    constructor(config = {}) {
        this.config = {
            // Encryption settings
            algorithm: 'aes-256-gcm',
            keyDerivationIterations: 100000,
            
            // Storage settings
            keyStorePath: config.keyStorePath || './secure-keys',
            backupPath: config.backupPath || './key-backups',
            
            // Security settings
            masterKey: config.masterKey || process.env.MASTER_KEY_ENCRYPTION_KEY,
            requireMFA: config.requireMFA || false,
            
            // Rotation settings
            defaultRotationDays: config.defaultRotationDays || 90,
            warningDays: config.warningDays || 7
        };
        
        // Validate configuration
        this.validateConfig();
        
        // Initialize key storage
        this.keyMetadata = new Map();
        this.keyCache = new Map();
        this.accessLog = [];
        
        // Initialize storage directories
        this.initializeStorage();
        
        console.log('✅ Secure key manager initialized');
    }
    
    validateConfig() {
        /**
         * Validate key manager configuration
         */
        
        if (!this.config.masterKey || this.config.masterKey.length < 32) {
            throw new Error('Master encryption key must be at least 32 characters');
        }
        
        // Ensure master key has sufficient entropy
        const uniqueChars = new Set(this.config.masterKey).size;
        if (uniqueChars < 16) {
            console.warn('⚠️ Master key may have insufficient entropy');
        }
        
        console.log('✅ Key manager configuration validated');
    }
    
    async initializeStorage() {
        /**
         * Initialize secure storage directories
         */
        
        try {
            await fs.mkdir(this.config.keyStorePath, { recursive: true });
            await fs.mkdir(this.config.backupPath, { recursive: true });
            
            // Set restrictive permissions (Unix-like systems)
            if (process.platform !== 'win32') {
                await fs.chmod(this.config.keyStorePath, 0o700); // Owner read/write/execute only
                await fs.chmod(this.config.backupPath, 0o700);
            }
            
            console.log('✅ Secure storage directories initialized');
            
        } catch (error) {
            console.error('❌ Failed to initialize storage:', error);
            throw error;
        }
    }
    
    async storeAPIKey(keyId, keyValue, metadata = {}) {
        /**
         * Securely store an API key with encryption and metadata
         * 
         * Args:
         *   keyId: Unique identifier for the key (e.g., 'openai_production')
         *   keyValue: The actual API key value
         *   metadata: Additional information about the key
         */
        
        try {
            console.log(`🔐 Storing API key: ${keyId}`);
            
            // Validate key format
            this.validateKeyFormat(keyId, keyValue, metadata);
            
            // Encrypt the key value
            const encryptedKey = await this.encryptKey(keyValue);
            
            // Create key record
            const keyRecord = {
                id: keyId,
                encrypted_value: encryptedKey,
                metadata: {
                    ...metadata,
                    created_at: new Date().toISOString(),
                    last_rotated: new Date().toISOString(),
                    rotation_due: this.calculateRotationDate(metadata.rotation_days),
                    access_count: 0,
                    last_accessed: null
                },
                permissions: metadata.permissions || ['read'],
                environments: metadata.environments || ['development']
            };
            
            // Store encrypted key file
            const keyFilePath = path.join(this.config.keyStorePath, `${keyId}.key`);
            await fs.writeFile(keyFilePath, JSON.stringify(keyRecord, null, 2));
            
            // Set restrictive file permissions
            if (process.platform !== 'win32') {
                await fs.chmod(keyFilePath, 0o600); // Owner read/write only
            }
            
            // Store metadata in memory for quick access
            this.keyMetadata.set(keyId, keyRecord.metadata);
            
            // Log the storage action
            this.logKeyAccess(keyId, 'stored', { success: true });
            
            console.log(`✅ API key stored securely: ${keyId}`);
            
            return {
                keyId: keyId,
                stored: true,
                rotationDue: keyRecord.metadata.rotation_due
            };
            
        } catch (error) {
            console.error(`❌ Failed to store API key ${keyId}:`, error);
            this.logKeyAccess(keyId, 'store_failed', { error: error.message });
            throw error;
        }
    }
    
    async retrieveAPIKey(keyId, requesterInfo = {}) {
        /**
         * Securely retrieve and decrypt an API key
         * 
         * Args:
         *   keyId: Key identifier
         *   requesterInfo: Information about who/what is requesting the key
         */
        
        try {
            // Check if key exists
            const keyFilePath = path.join(this.config.keyStorePath, `${keyId}.key`);
            
            try {
                await fs.access(keyFilePath);
            } catch {
                throw new Error(`API key not found: ${keyId}`);
            }
            
            // Load key record
            const keyRecordData = await fs.readFile(keyFilePath, 'utf8');
            const keyRecord = JSON.parse(keyRecordData);
            
            // Check permissions
            if (!this.checkKeyPermissions(keyRecord, requesterInfo)) {
                throw new Error(`Access denied for key: ${keyId}`);
            }
            
            // Check if key is expired or needs rotation
            const rotationCheck = this.checkRotationStatus(keyRecord.metadata);
            if (rotationCheck.expired) {
                throw new Error(`API key expired: ${keyId}`);
            }
            
            if (rotationCheck.warningNeeded) {
                console.warn(`⚠️ API key ${keyId} needs rotation in ${rotationCheck.daysUntilRotation} days`);
            }
            
            // Decrypt the key value
            const decryptedKey = await this.decryptKey(keyRecord.encrypted_value);
            
            // Update access metadata
            await this.updateKeyAccessMetadata(keyId, keyRecord);
            
            // Log the access
            this.logKeyAccess(keyId, 'retrieved', { 
                success: true,
                requester: requesterInfo.service || 'unknown'
            });
            
            console.log(`🔓 API key retrieved: ${keyId}`);
            
            return {
                keyValue: decryptedKey,
                metadata: keyRecord.metadata,
                rotationWarning: rotationCheck.warningNeeded
            };
            
        } catch (error) {
            console.error(`❌ Failed to retrieve API key ${keyId}:`, error);
            this.logKeyAccess(keyId, 'retrieve_failed', { 
                error: error.message,
                requester: requesterInfo.service || 'unknown'
            });
            throw error;
        }
    }
    
    validateKeyFormat(keyId, keyValue, metadata) {
        /**
         * Validate API key format and metadata
         */
        
        // Check key ID format
        if (!keyId || typeof keyId !== 'string' || keyId.length < 3) {
            throw new Error('Key ID must be a string with at least 3 characters');
        }
        
        // Check for valid key ID pattern (alphanumeric, hyphens, underscores)
        if (!/^[a-zA-Z0-9_-]+$/.test(keyId)) {
            throw new Error('Key ID contains invalid characters');
        }
        
        // Check key value
        if (!keyValue || typeof keyValue !== 'string' || keyValue.length < 10) {
            throw new Error('Key value must be a string with at least 10 characters');
        }
        
        // Validate key format based on service
        const service = metadata.service || 'unknown';
        this.validateServiceKeyFormat(service, keyValue);
        
        console.log(`✅ Key format validated: ${keyId} (${service})`);
    }
    
    validateServiceKeyFormat(service, keyValue) {
        /**
         * Validate key format for specific services
         */
        
        const servicePatterns = {
            openai: /^sk-[a-zA-Z0-9]{48}$/,
            anthropic: /^sk-ant-[a-zA-Z0-9-]{95}$/,
            slack: /^xoxb-[0-9]+-[0-9]+-[a-zA-Z0-9]+$/,
            discord: /^[A-Za-z0-9_-]{59}$/,
            github: /^ghp_[a-zA-Z0-9]{36}$/
        };
        
        const pattern = servicePatterns[service.toLowerCase()];
        
        if (pattern && !pattern.test(keyValue)) {
            console.warn(`⚠️ Key format doesn't match expected pattern for ${service}`);
            // Don't throw error - patterns may change, but log the warning
        }
    }
    
    async encryptKey(keyValue) {
        /**
         * Encrypt API key using AES-256-GCM
         */
        
        const algorithm = this.config.algorithm;
        const salt = crypto.randomBytes(32);
        const iv = crypto.randomBytes(16);
        
        // Derive encryption key from master key
        const key = crypto.pbkdf2Sync(
            this.config.masterKey,
            salt,
            this.config.keyDerivationIterations,
            32,
            'sha256'
        );
        
        // Create cipher
        const cipher = crypto.createCipher(algorithm, key);
        
        // Encrypt the key value
        let encrypted = cipher.update(keyValue, 'utf8', 'hex');
        encrypted += cipher.final('hex');
        
        // Get authentication tag
        const tag = cipher.getAuthTag();
        
        return {
            encrypted: encrypted,
            salt: salt.toString('hex'),
            iv: iv.toString('hex'),
            tag: tag.toString('hex'),
            algorithm: algorithm
        };
    }
    
    async decryptKey(encryptedKeyData) {
        /**
         * Decrypt API key
         */
        
        const algorithm = encryptedKeyData.algorithm;
        const salt = Buffer.from(encryptedKeyData.salt, 'hex');
        const iv = Buffer.from(encryptedKeyData.iv, 'hex');
        const tag = Buffer.from(encryptedKeyData.tag, 'hex');
        
        // Derive decryption key
        const key = crypto.pbkdf2Sync(
            this.config.masterKey,
            salt,
            this.config.keyDerivationIterations,
            32,
            'sha256'
        );
        
        // Create decipher
        const decipher = crypto.createDecipher(algorithm, key);
        decipher.setAuthTag(tag);
        
        // Decrypt the key value
        let decrypted = decipher.update(encryptedKeyData.encrypted, 'hex', 'utf8');
        decrypted += decipher.final('utf8');
        
        return decrypted;
    }
    
    calculateRotationDate(rotationDays) {
        /**
         * Calculate when key should be rotated
         */
        
        const days = rotationDays || this.config.defaultRotationDays;
        const rotationDate = new Date();
        rotationDate.setDate(rotationDate.getDate() + days);
        
        return rotationDate.toISOString();
    }
    
    checkRotationStatus(keyMetadata) {
        /**
         * Check if key needs rotation
         */
        
        const now = new Date();
        const rotationDue = new Date(keyMetadata.rotation_due);
        const warningDate = new Date(rotationDue);
        warningDate.setDate(warningDate.getDate() - this.config.warningDays);
        
        return {
            expired: now > rotationDue,
            warningNeeded: now > warningDate,
            daysUntilRotation: Math.ceil((rotationDue - now) / (1000 * 60 * 60 * 24))
        };
    }
    
    logKeyAccess(keyId, action, details = {}) {
        /**
         * Log all key access for security auditing
         */
        
        const logEntry = {
            timestamp: new Date().toISOString(),
            keyId: keyId,
            action: action,
            details: details,
            requestId: crypto.randomUUID()
        };
        
        this.accessLog.push(logEntry);
        
        // In production, send to secure logging service
        console.log(`📊 Key access logged: ${keyId} - ${action}`);
        
        // Keep log size manageable (in production, use external logging)
        if (this.accessLog.length > 1000) {
            this.accessLog = this.accessLog.slice(-500);
        }
    }
}

Key Management Explanation:

Encryption at Rest: API keys are never stored in plain text. Even if someone gains access to your key files, they can't read the keys without the master encryption key.

Access Logging: Every key access is logged with timestamp, requester information, and success/failure status. This creates an audit trail for security investigations.

Metadata Tracking: We track when keys were created, last accessed, and when they need rotation. This enables proactive key management.

---

Step 3: Automated Key Rotation

Regular key rotation limits the damage from compromised keys and is required by many security frameworks.

Key Rotation Strategy

Why Rotate API Keys:

• Limit Exposure Time: Even if a key is compromised, it becomes useless after rotation
• Compliance Requirements: Many security frameworks require regular key rotation
• Proactive Security: Rotation before compromise is better than reaction after

Rotation Challenges for AI Agents:

• Zero-Downtime Rotation: AI agents must continue operating during key rotation
• Multiple Service Coordination: Some keys are used across multiple services
• Rollback Capability: If new keys fail, you need to quickly revert to old keys

Automated Rotation Implementation

// security/key-rotation.js

class KeyRotationManager {
    constructor(keyManager, serviceClients) {
        this.keyManager = keyManager;
        this.serviceClients = serviceClients; // Map of service name to client
        
        // Rotation state tracking
        this.rotationInProgress = new Set();
        this.rotationHistory = [];
        
        // Start rotation monitoring
        this.startRotationMonitoring();
        
        console.log('✅ Key rotation manager initialized');
    }
    
    startRotationMonitoring() {
        /**
         * Start automated monitoring for keys that need rotation
         */
        
        // Check for keys needing rotation every hour
        setInterval(() => {
            this.checkKeysForRotation();
        }, 60 * 60 * 1000);
        
        console.log('🔄 Key rotation monitoring started');
    }
    
    async checkKeysForRotation() {
        /**
         * Check all keys and rotate those that are due
         */
        
        console.log('🔍 Checking keys for rotation...');
        
        const keysNeedingRotation = [];
        
        // Check each key's rotation status
        for (const [keyId, metadata] of this.keyManager.keyMetadata) {
            const rotationStatus = this.keyManager.checkRotationStatus(metadata);
            
            if (rotationStatus.expired) {
                keysNeedingRotation.push({
                    keyId: keyId,
                    priority: 'critical',
                    daysOverdue: Math.abs(rotationStatus.daysUntilRotation)
                });
            } else if (rotationStatus.warningNeeded) {
                keysNeedingRotation.push({
                    keyId: keyId,
                    priority: 'warning',
                    daysUntilDue: rotationStatus.daysUntilRotation
                });
            }
        }
        
        // Process rotations
        for (const keyInfo of keysNeedingRotation) {
            if (keyInfo.priority === 'critical') {
                console.error(`🚨 CRITICAL: Key ${keyInfo.keyId} is ${keyInfo.daysOverdue} days overdue for rotation`);
                await this.rotateKey(keyInfo.keyId, { urgent: true });
            } else {
                console.warn(`⚠️ Key ${keyInfo.keyId} needs rotation in ${keyInfo.daysUntilDue} days`);
                // Schedule rotation for non-critical keys
                await this.scheduleKeyRotation(keyInfo.keyId);
            }
        }
        
        if (keysNeedingRotation.length === 0) {
            console.log('✅ All keys are current - no rotation needed');
        }
    }
    
    async rotateKey(keyId, options = {}) {
        /**
         * Perform zero-downtime key rotation
         * 
         * Args:
         *   keyId: Key to rotate
         *   options: Rotation options (urgent, test_mode, etc.)
         */
        
        if (this.rotationInProgress.has(keyId)) {
            console.warn(`⚠️ Rotation already in progress for key: ${keyId}`);
            return;
        }
        
        this.rotationInProgress.add(keyId);
        
        const rotationId = crypto.randomUUID();
        const startTime = Date.now();
        
        console.log(`🔄 Starting key rotation: ${keyId} (${rotationId})`);
        
        try {
            // Step 1: Generate new key (service-specific)
            const newKeyValue = await this.generateNewKey(keyId, options);
            
            // Step 2: Test new key
            const testResult = await this.testNewKey(keyId, newKeyValue);
            if (!testResult.success) {
                throw new Error(`New key test failed: ${testResult.error}`);
            }
            
            // Step 3: Store new key with temporary ID
            const tempKeyId = `${keyId}_new_${Date.now()}`;
            await this.keyManager.storeAPIKey(tempKeyId, newKeyValue, {
                service: this.getKeyService(keyId),
                temporary: true,
                replaces: keyId
            });
            
            // Step 4: Update services to use new key (gradual rollout)
            await this.updateServicesWithNewKey(keyId, tempKeyId, options);
            
            // Step 5: Monitor for issues
            const monitoringResult = await this.monitorKeyPerformance(tempKeyId, 300000); // 5 minutes
            
            if (!monitoringResult.success) {
                // Rollback to old key
                console.error(`❌ New key performance issues, rolling back: ${keyId}`);
                await this.rollbackKeyRotation(keyId, tempKeyId);
                throw new Error(`Key rotation failed: ${monitoringResult.error}`);
            }
            
            // Step 6: Replace old key with new key
            await this.finalizeKeyRotation(keyId, tempKeyId);
            
            // Step 7: Securely dispose of old key
            await this.disposeOldKey(keyId);
            
            const rotationTime = Date.now() - startTime;
            
            // Record successful rotation
            this.rotationHistory.push({
                keyId: keyId,
                rotationId: rotationId,
                timestamp: new Date().toISOString(),
                duration: rotationTime,
                success: true
            });
            
            console.log(`✅ Key rotation completed: ${keyId} (${rotationTime}ms)`);
            
            return {
                keyId: keyId,
                rotationId: rotationId,
                success: true,
                duration: rotationTime
            };
            
        } catch (error) {
            console.error(`❌ Key rotation failed: ${keyId}`, error);
            
            // Record failed rotation
            this.rotationHistory.push({
                keyId: keyId,
                rotationId: rotationId,
                timestamp: new Date().toISOString(),
                duration: Date.now() - startTime,
                success: false,
                error: error.message
            });
            
            throw error;
            
        } finally {
            this.rotationInProgress.delete(keyId);
        }
    }
    
    async testNewKey(keyId, newKeyValue) {
        /**
         * Test new API key before putting it into production
         */
        
        const service = this.getKeyService(keyId);
        const serviceClient = this.serviceClients.get(service);
        
        if (!serviceClient) {
            return { success: false, error: `No test client available for service: ${service}` };
        }
        
        try {
            console.log(`🧪 Testing new key for service: ${service}`);
            
            // Perform service-specific key test
            const testResult = await serviceClient.testKey(newKeyValue);
            
            if (testResult.success) {
                console.log(`✅ New key test passed for ${service}`);
                return { success: true };
            } else {
                return { success: false, error: testResult.error };
            }
            
        } catch (error) {
            return { success: false, error: error.message };
        }
    }
    
    getKeyService(keyId) {
        /**
         * Determine service type from key ID
         */
        
        const serviceMap = {
            openai: ['openai', 'gpt'],
            anthropic: ['anthropic', 'claude'],
            slack: ['slack'],
            discord: ['discord'],
            github: ['github', 'git']
        };
        
        const keyIdLower = keyId.toLowerCase();
        
        for (const [service, keywords] of Object.entries(serviceMap)) {
            if (keywords.some(keyword => keyIdLower.includes(keyword))) {
                return service;
            }
        }
        
        return 'unknown';
    }
}

Key Rotation Explanation:

Zero-Downtime Strategy: New keys are tested and gradually rolled out before old keys are disabled, ensuring continuous service availability.

Automated Testing: New keys are automatically tested with the target service before being put into production use.

Rollback Capability: If issues are detected with new keys, the system can quickly revert to the previous working keys.

Monitoring Integration: Key performance is monitored after rotation to detect any issues early.

---

Step 4: Multi-Environment Key Management

AI agents typically operate across multiple environments, each requiring different security levels and key management strategies.

Environment-Specific Key Strategies

Development Environment

• Lower security requirements (faster development)
• Test API keys with limited quotas
• Shared keys among team members (with proper access controls)
• Frequent rotation for testing rotation procedures

Staging Environment

• Production-like security for realistic testing
• Separate keys from production
• Automated testing of key rotation procedures
• Performance testing with production-equivalent keys

Production Environment

• Maximum security with all protections enabled
• Unique keys never used in other environments
• Strict access controls and comprehensive monitoring
• Formal rotation procedures with approval workflows

Environment Configuration

// config/environment-config.js

class EnvironmentKeyConfig {
    constructor() {
        this.environments = {
            development: {
                security_level: 'standard',
                rotation_days: 30,
                require_mfa: false,
                allow_shared_keys: true,
                monitoring_level: 'basic',
                
                // Development-specific settings
                allow_test_keys: true,
                quota_limits: 'low',
                access_logging: 'minimal'
            },
            
            staging: {
                security_level: 'high',
                rotation_days: 60,
                require_mfa: true,
                allow_shared_keys: false,
                monitoring_level: 'detailed',
                
                // Staging-specific settings
                allow_test_keys: false,
                quota_limits: 'medium',
                access_logging: 'full'
            },
            
            production: {
                security_level: 'maximum',
                rotation_days: 90,
                require_mfa: true,
                allow_shared_keys: false,
                monitoring_level: 'comprehensive',
                
                // Production-specific settings
                allow_test_keys: false,
                quota_limits: 'high',
                access_logging: 'full',
                require_approval: true,
                backup_keys: true
            }
        };
    }
    
    getEnvironmentConfig(environment) {
        /**
         * Get configuration for specific environment
         */
        
        const config = this.environments[environment];
        if (!config) {
            throw new Error(`Unknown environment: ${environment}`);
        }
        
        return config;
    }
    
    validateKeyForEnvironment(keyId, keyMetadata, environment) {
        /**
         * Validate that key meets environment security requirements
         */
        
        const envConfig = this.getEnvironmentConfig(environment);
        const issues = [];
        
        // Check rotation requirements
        const rotationStatus = this.checkRotationStatus(keyMetadata);
        if (rotationStatus.expired && envConfig.security_level === 'maximum') {
            issues.push('Key is expired and cannot be used in production');
        }
        
        // Check access requirements
        if (envConfig.require_mfa && !keyMetadata.mfa_protected) {
            issues.push('Key requires MFA protection for this environment');
        }
        
        // Check sharing restrictions
        if (!envConfig.allow_shared_keys && keyMetadata.shared) {
            issues.push('Shared keys not allowed in this environment');
        }
        
        return {
            valid: issues.length === 0,
            issues: issues,
            environment: environment,
            security_level: envConfig.security_level
        };
    }
}

Environment Management Benefits:

Security Isolation: Production keys are completely separate from development keys, preventing accidental exposure.

Appropriate Security Levels: Each environment has security measures appropriate to its risk level and usage patterns.

Testing Safety: Development and staging environments can test key rotation and security procedures without affecting production.

---

Step 5: Key Usage Monitoring and Anomaly Detection

Monitoring API key usage helps detect compromised keys and optimize costs.

Usage Monitoring System

// monitoring/key-usage-monitor.js

class KeyUsageMonitor {
    constructor(keyManager) {
        this.keyManager = keyManager;
        
        // Usage tracking
        this.usageMetrics = new Map();
        this.anomalyThresholds = {
            requestsPerHour: 1000,
            costPerDay: 100,
            unusualLocations: true,
            offHoursUsage: true
        };
        
        // Start monitoring
        this.startUsageMonitoring();
        
        console.log('✅ Key usage monitoring initialized');
    }
    
    recordKeyUsage(keyId, usageData) {
        /**
         * Record API key usage for monitoring
         * 
         * Args:
         *   keyId: Key identifier
         *   usageData: Usage information (tokens, cost, location, etc.)
         */
        
        const timestamp = new Date().toISOString();
        
        // Get or create usage metrics for this key
        if (!this.usageMetrics.has(keyId)) {
            this.usageMetrics.set(keyId, {
                totalRequests: 0,
                totalTokens: 0,
                totalCost: 0,
                locations: new Set(),
                hourlyUsage: new Map(),
                dailyUsage: new Map()
            });
        }
        
        const metrics = this.usageMetrics.get(keyId);
        
        // Update metrics
        metrics.totalRequests++;
        metrics.totalTokens += usageData.tokens || 0;
        metrics.totalCost += usageData.cost || 0;
        
        if (usageData.location) {
            metrics.locations.add(usageData.location);
        }
        
        // Track hourly usage
        const hour = new Date().getHours();
        metrics.hourlyUsage.set(hour, (metrics.hourlyUsage.get(hour) || 0) + 1);
        
        // Check for anomalies
        this.checkForAnomalies(keyId, usageData, metrics);
    }
    
    checkForAnomalies(keyId, usageData, metrics) {
        /**
         * Check for unusual usage patterns that might indicate compromise
         */
        
        const anomalies = [];
        
        // Check request rate
        const currentHour = new Date().getHours();
        const requestsThisHour = metrics.hourlyUsage.get(currentHour) || 0;
        
        if (requestsThisHour > this.anomalyThresholds.requestsPerHour) {
            anomalies.push({
                type: 'high_request_rate',
                severity: 'medium',
                details: `${requestsThisHour} requests in current hour`
            });
        }
        
        // Check for unusual locations
        if (this.anomalyThresholds.unusualLocations && usageData.location) {
            const knownLocations = Array.from(metrics.locations);
            if (knownLocations.length > 0 && !knownLocations.includes(usageData.location)) {
                anomalies.push({
                    type: 'unusual_location',
                    severity: 'high',
                    details: `New location: ${usageData.location}`
                });
            }
        }
        
        // Alert on anomalies
        if (anomalies.length > 0) {
            this.handleUsageAnomalies(keyId, anomalies);
        }
    }
}

---

## What You've Accomplished

You now have **enterprise-grade API key management** for your AI agent:

- ✅ **Secure Storage**: Keys encrypted at rest with proper access controls
- ✅ **Automated Rotation**: Zero-downtime key rotation with testing and rollback
- ✅ **Multi-Environment Support**: Proper isolation between dev, staging, and production
- ✅ **Usage Monitoring**: Anomaly detection and cost tracking
- ✅ **Compliance Framework**: Audit trails and security reporting
- ✅ **Incident Response**: Procedures for handling compromised keys

### Key Security Features Implemented:

1. **Encryption at Rest**: All keys stored with AES-256-GCM encryption
2. **Access Control**: Fine-grained permissions and environment isolation
3. **Automated Rotation**: Scheduled rotation with performance monitoring
4. **Usage Monitoring**: Real-time anomaly detection and alerting
5. **Audit Logging**: Complete trail of all key access and operations

---

## What's Next?

In **Part 5: Security Monitoring & Auditing**, you'll learn:

- Automated vulnerability scanning for your entire AI agent system
- Real-time threat detection and response
- Compliance reporting for regulatory requirements
- Incident response procedures for security breaches

### Quick Setup Commands

```bash
# Install required packages
npm install crypto fs-extra

# Generate master encryption key
node -e "console.log(require('crypto').randomBytes(64).toString('base64url'))"

# Initialize key manager
node scripts/initialize-key-manager.js

# Test key rotation
node scripts/test-key-rotation.js

---

Security Checklist

Before deploying to production:

• Master Key: Securely generated and stored master encryption key
• Environment Separation: Different keys for dev, staging, production
• Rotation Schedule: Automated rotation configured and tested
• Access Controls: Proper permissions and MFA where required
• Monitoring: Usage monitoring and anomaly detection active
• Backup Strategy: Secure backup of key metadata and rotation procedures
• Incident Response: Documented procedures for key compromise
• Compliance: Audit logging and reporting configured

---

Additional Resources

• NIST Key Management Guidelines: csrc.nist.gov/publications/detail/sp/800-57-part-1/rev-5/final
• OWASP Key Management: owasp.org/www-project-cheat-sheets/cheatsheets/Key_Management_Cheat_Sheet.html
• AWS Secrets Manager: docs.aws.amazon.com/secretsmanager/
• Azure Key Vault: docs.microsoft.com/en-us/azure/key-vault/

Your AI agent now has enterprise-grade API key security that protects against key compromise and ensures compliance with security frameworks. Continue to Part 5: Security Monitoring & Auditing to complete your comprehensive security implementation!

---

Tutorial Navigation

• ← Part 3: Data Privacy & Encryption
• Part 4: Secure API Key Management (Current)
• Part 5: Security Monitoring & Auditing →

---

This tutorial is part of our comprehensive AI Agent Security series. Proper API key management is crucial for maintaining security and controlling costs - implement these practices, and your AI agent will be protected against key-related security incidents.