AI应用的安全工程从威胁建模到防护前言我们的产品上线后遇到了各种安全问题Prompt 注入、数据泄露、API 滥用...今天分享我们是如何建立安全工程体系的。一、威胁建模1.1 威胁识别class ThreatModeling: def identify(self) - list: 识别威胁 return [ {type: prompt_injection, severity: high}, {type: data_leakage, severity: high}, {type: api_abuse, severity: medium}, {type: model_manipulation, severity: medium} ]1.2 风险评估class RiskAssessment: def assess(self, threats: list) - dict: 评估风险 prioritized sorted(threats, keylambda x: x[severity], reverseTrue) return { critical: [t for t in prioritized if t[severity] high], medium: [t for t in prioritized if t[severity] medium] }二、安全防护2.1 输入验证class InputValidation: def validate(self, input_data: str) - dict: 验证输入 checks [ {name: length, passed: len(input_data) 5000}, {name: patterns, passed: not self._contains_suspicious(input_data)}, {name: sanitization, passed: True} ] return {valid: all(c[passed] for c in checks), checks: checks}2.2 输出过滤class OutputFiltering: def filter(self, output: str) - str: 过滤输出 # 移除敏感信息 output self._remove_personal_info(output) # 检查内容安全性 if self._contains_harmful(output): return 抱歉无法生成相关内容 return output三、API 安全3.1 认证授权class Authentication: def authenticate(self, token: str) - dict: 认证用户 return { user_id: 123, role: user, valid: True } def authorize(self, user_id: str, resource: str) - bool: 授权检查 permissions {user: [read], admin: [read, write]} return resource in permissions.get(self._get_role(user_id), [])3.2 限流保护class RateLimiting: def __init__(self): self.limits {free: 100, pro: 1000} def check(self, user_id: str, plan: str) - bool: 检查限流 current self._get_request_count(user_id) return current self.limits.get(plan, 100)四、数据安全4.1 数据加密class DataEncryption: def encrypt(self, data: str) - str: 加密数据 from cryptography.fernet import Fernet key Fernet.generate_key() f Fernet(key) return f.encrypt(data.encode()).decode() def decrypt(self, encrypted: str) - str: 解密数据 from cryptography.fernet import Fernet key self._get_key() f Fernet(key) return f.decrypt(encrypted.encode()).decode()4.2 访问控制class AccessControl: def check(self, user_id: str, resource: str) - bool: 检查访问权限 return resource own_data or self._is_admin(user_id)五、安全监控5.1 异常检测class AnomalyDetection: def detect(self, activity: dict) - dict: 检测异常 anomalies [] if activity[requests_per_minute] 100: anomalies.append(请求频率异常) if activity[unusual_pattern]: anomalies.append(行为模式异常) return {anomalies: anomalies, risk_level: high if anomalies else low}5.2 告警机制class Alerting: def trigger(self, alert: dict): 触发告警 return { type: alert[type], severity: alert[severity], message: alert[message], actions: [通知安全团队, 临时限流] }六、最佳实践6.1 安全原则✅纵深防御多层安全防护✅最小权限只给必要权限✅默认安全安全配置默认开启✅持续监控实时监控安全事件6.2 常见误区❌安全事后等出事再做安全❌单点依赖依赖单一安全措施❌忽视测试不做安全测试❌过度防护影响用户体验七、总结安全工程是 AI 应用的基石。关键在于威胁建模提前识别风险多层防护建立纵深防御持续监控及时发现问题持续改进根据反馈优化记住安全不是一次性工作是持续的过程。
AI应用的安全工程：从威胁建模到防护
AI应用的安全工程从威胁建模到防护前言我们的产品上线后遇到了各种安全问题Prompt 注入、数据泄露、API 滥用...今天分享我们是如何建立安全工程体系的。一、威胁建模1.1 威胁识别class ThreatModeling: def identify(self) - list: 识别威胁 return [ {type: prompt_injection, severity: high}, {type: data_leakage, severity: high}, {type: api_abuse, severity: medium}, {type: model_manipulation, severity: medium} ]1.2 风险评估class RiskAssessment: def assess(self, threats: list) - dict: 评估风险 prioritized sorted(threats, keylambda x: x[severity], reverseTrue) return { critical: [t for t in prioritized if t[severity] high], medium: [t for t in prioritized if t[severity] medium] }二、安全防护2.1 输入验证class InputValidation: def validate(self, input_data: str) - dict: 验证输入 checks [ {name: length, passed: len(input_data) 5000}, {name: patterns, passed: not self._contains_suspicious(input_data)}, {name: sanitization, passed: True} ] return {valid: all(c[passed] for c in checks), checks: checks}2.2 输出过滤class OutputFiltering: def filter(self, output: str) - str: 过滤输出 # 移除敏感信息 output self._remove_personal_info(output) # 检查内容安全性 if self._contains_harmful(output): return 抱歉无法生成相关内容 return output三、API 安全3.1 认证授权class Authentication: def authenticate(self, token: str) - dict: 认证用户 return { user_id: 123, role: user, valid: True } def authorize(self, user_id: str, resource: str) - bool: 授权检查 permissions {user: [read], admin: [read, write]} return resource in permissions.get(self._get_role(user_id), [])3.2 限流保护class RateLimiting: def __init__(self): self.limits {free: 100, pro: 1000} def check(self, user_id: str, plan: str) - bool: 检查限流 current self._get_request_count(user_id) return current self.limits.get(plan, 100)四、数据安全4.1 数据加密class DataEncryption: def encrypt(self, data: str) - str: 加密数据 from cryptography.fernet import Fernet key Fernet.generate_key() f Fernet(key) return f.encrypt(data.encode()).decode() def decrypt(self, encrypted: str) - str: 解密数据 from cryptography.fernet import Fernet key self._get_key() f Fernet(key) return f.decrypt(encrypted.encode()).decode()4.2 访问控制class AccessControl: def check(self, user_id: str, resource: str) - bool: 检查访问权限 return resource own_data or self._is_admin(user_id)五、安全监控5.1 异常检测class AnomalyDetection: def detect(self, activity: dict) - dict: 检测异常 anomalies [] if activity[requests_per_minute] 100: anomalies.append(请求频率异常) if activity[unusual_pattern]: anomalies.append(行为模式异常) return {anomalies: anomalies, risk_level: high if anomalies else low}5.2 告警机制class Alerting: def trigger(self, alert: dict): 触发告警 return { type: alert[type], severity: alert[severity], message: alert[message], actions: [通知安全团队, 临时限流] }六、最佳实践6.1 安全原则✅纵深防御多层安全防护✅最小权限只给必要权限✅默认安全安全配置默认开启✅持续监控实时监控安全事件6.2 常见误区❌安全事后等出事再做安全❌单点依赖依赖单一安全措施❌忽视测试不做安全测试❌过度防护影响用户体验七、总结安全工程是 AI 应用的基石。关键在于威胁建模提前识别风险多层防护建立纵深防御持续监控及时发现问题持续改进根据反馈优化记住安全不是一次性工作是持续的过程。
